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Introduction 

A reductive Q-simple algebraic group G is of hermitian type, if the symmetric space D defined by 
(jr(M) is a hermitian symmetric space. A discrete subgroup T C G{Q) is arithmetic, if for some faithful 
rational representation p : G — > GL{V), and for some lattice Vz C Vq, T is commensurable with 
p~^{GL{Vz))- A non-compact hermitian symmetric space is holomorphically equivalent to a bounded 
symmetric domain. If this is the case, T acts on T> preserving the natural Bergmann metric, and 
Xy = r\P is, if r is torsion-free, a complex manifold, in general not compact (it is compact exactly 
when G is anisotropic). We call spaces arithmetic quotients of bounded symmetric domains even 
when r has torsion; it is known that in this case Xr is a IZ-manifold (in the sense of Satake), locally 
the quotient of a smooth space by a finite group action. There is a natural compactification of Xy, 
the Satake compactification X^, which has the property: the complement X^ — X-p is a finite disjoint 
union of arithmetic quotients of bounded symmetric domains of lower dimension. A natural problem 
in this respect is to consider, in addition to the data above, a symmetric subdomain V d T) which 
has the property that the restriction of the action of T to V is discrete, say by a discrete subgroup 
r', resulting in a commutative square 

V ^ V 

i i 
Xy' = T'\V' ^ T\D = Xr- 

It is spaces arising as X-p' that we will refer to as modular subvarieties. This situation ensuses in 
particular if C G is a reductive subgroup of hermitian type; then the symmetric space Pat of A'^ has 
a holomorphic symmetric embedding P^r C P. As a matter of notation, we refer to such subgroups 
as symmetric subgroups. This explains the title and describes the topic of this paper. 

We are in fact concerned with a very special set of modular subvarieties which have very special 
behavior at the cusps. Upstairs in the universal covers (in the domains D' and P), this behavior 
defines a notion of incidence, implying a relation between (real) parabolic and symmetric subgroups. 
Conversely, one can define a notion of incidence, group theoretically, between parabolic and symmetric 
subgroups P and A^, which implies the desired behavior of the subdomain 2?7v (the V in the notations 
above) near the cusp F which corresponds to P. It is the geometric point of view we will adapt in this 
paper. Starting with a rational boundary component F (respectively the corresponding parabolic P), 
we define what it means for F and a symmetric subdomain T>jsr (respectively the parabolic P and a 
symmetric subgroup A^) to be incident. For F and Pat this is easy to formulate: T>n and F are incident 



if is a rational boundary component of Pat and maximal with this property. For the subgroups 
and P the condition is more complicated to formulate, but consists essentially in a maximality 
condition plus the condition N = Ni x N2, and A^i is a hermitain Levi factor of P (if dim(F) > 0) or 
just that is irreducible such that F is a boundary component of "Dn (if dim(i<') = 0), see below for 
details. 

This gives rise to a notion of incidence for Q-subgroups P and N, by insisting that incidence holds 
as above for the groups of real points P(M) and N{M). Everything being defined over Q, one can 
proceed to form an arithmetic quotient by an arithmetic subgroup T C G{Q_), and T will act naturally 
both on F and on P^r. The quotients will be a boundary variety W and a modular subvariety X-p' 
as above, and the notion of incidence now becomes W C X^,, where X^, C X^ is the embedding of 



Baily-Borel embeddings described in [32]. Thus the problem has two aspects 

a) The question of the existence of Q-subgroups N which are incident with a given P, and 

b) the action of T on the set of such subgroups. 



The first point a) was treated in detail in [HI] and [[H2([ ; this paper contains a preliminary study of b). 

Similar questions have been asked and answered for parabolic subgroups, but the situation for 
symmetric subgroups is totally different. The basic cause for this is the following elementary fact: if 
two Q-parabolics are conjugate (over the algebraic closure), then they are in fact Q-conjugate, i.e., 
conjugate by an element of G(Q), whereas the corresponding statement for symmetric subgroups is 
totally wrong. The reason behind this is the very fundamental fact that the homogenous spaces G/P 
are projective for parabolic subgroups P, while G/N is affine for a reductive subgroup A^, a property 
which in fact characterizes reductive subgroups A^, as was proved in ]BHC| . A consequence of this 
is that it holds for the parabolic subgroup P that G(Q)/P(Q) = (G/P)(Q), while for symmetric 
subgroups A^ only the inclusion G{Q)/N{Q) C {G/N){Q) holds. The elements of the left-hand side 
are those subgroups A^'(Q) C G(Q) which are G(Q)-conjugate to A^(Q), while on the right-hand side 
we have those G-conjugates of A^ which are defined over Q. Considering only the former leads to 
the definition of rational symmetric subgroups (Definition |4.1| ). From this point on we restrict our 
attention to symmetric subgroups A^ which are incident with a given parabolic P (or the other way 
around, parabolic subgroups P which are incident with a given symmetric subgroup A^). This means 
we consider for each b £ {1, . . . ,s}, s = rankjjG, a fixed isomorphism class (with the exception b = s 
for the two exceptional domains, for which there are three, resp. two such isomorphism classes) of the 
groups of real points A^(R) for subgroups A'^ (conjugate to a fixed symmetric subgroup A'b incident with 
Pbi a standard parabolic), and consider pairs {N,P) of incident symmetric and parabolic subgroups. 
We consider rational symmetric subgroups N representing points of G(Q)/A'^b(Q)) i-e., A^ is G{Q)- 
conjugate to A^b- However, these turn out to still be too many such subgroups in the sense that there 
are in general infinitely many F-equivalence classes for any F. We are led to introduce the notion of F- 
integral symmetric subgroups (Definition [4.3| ), of which there are finitely many F-orbits. To formulate 
this we assume dim(F) > and let A^b = A^b,i x A''b,2 (resp. N = Ni x N2) be the decomposition of 
A^b (resp. of A^) mentioned above. Then the definition in this case is: 

N = gN^g-^ is integral ^ gNig-^ n F = g{N^,i n r)g~\ 

and it depends in fact on the choice of A^b (that is, on the choice of maximal Q-split torus and order on 
it) as well as on F. (For dim(i<') = 0, the condition is simply gNg~^ n F = ^(A^b H F)*^""'^.) Fixing this 
data A'b and F leads to a finite number of F'-equivalence classes of F-integral symmetric subgroups 
conjugate to A^b for any arithmetic group F' C G(Q), which we show by utilizing the basic finiteness 



result of [[BHq] 
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We now consider F-integral symmetric subgroups N and arbitrary arithmetic subgroups T' C G(Q), 
let = N CiV and consider the corresponding modular subvarieties they define, C X-p'] we 



call these integral modular subvarieties (Definition |6.2| ). As mentioned above, the inclusion extends 
to the Baily-Borel embeddings Xp, C X^, . We now take F to be Gz for some rational representation 

p : G — > GL{V), that is T = p^^{GL{Vz)) for some Z-structure Vz on V. Putting all the pieces 
together yields our main result, which we now formulate. For this we refer to the notations i^f,(T'), b 



l,...,s and //f,(F,F'), b = l,...,s of Definition 4.12 and respectively, for the number of b*^ 
boundary varieties and the number of b*"^ integral modular subvarieties, respectively. We let Wf,^,, b = 
1, . . . , s, i = 1, . . . , f;,(F') be the corresponding boundary varieties on the Satake compactification, 
Yfej, b = 1,. . . ,s j = 1, . . . ,iJ,f,(T,T') the corresponding F-integral modular varieties, everything on 
the arithmetic quotient Xp/. Then the main result of the paper is the following. 

Theorem 0.1 Let F be as above, T' C G'(Q) arithmetic, and X-p' C X^, the Satake compactification, 
Xp, — X]-i = "^ifiWb^i. Then H := X^^j^bj is a complete (finite, non-empty) set of T' -equivalence 
classes ofT-integral modular subvarieties, such that for each Wh^i, there is at least one Y^j incident 
to Wb,i. 

This gives us a well-defined, non-empty, finite set of subvarieties of the Baily-Borel embedding 
Xp, C for any subgroup F' C F of finite index. Furthermore these have a prescribed behavior near 
the cusps. For example, if / : I? — > C is a modular form whose zero divisor Df on X^, contains the 
union of the integral modular subvarieties, then / is a cusp form for F'. 

In the case that the arithmetic quotient X^ is a moduli space of abelian varieties with some Pel 
structure, it is not difficult to see the moduli interpretation of the modular subvarieties X-pj^ determined 



by rational symmetric subgroups. See Example 4.2, where an interesting case is discussed in more 



detail. That is the case G = Sp{A,Q), and it turns out that in this case many modular subvarieties 
which come from Q-groups N conjugate to the standard one have a nice moduli interpretation: they 
parameterize abelian varieties (surfaces in this case) with real multiplication by a real quadratic field k. 
The group A'' is G(Q)-conjugate by the standard one Ni precisely when the field k splits into a product 
Q X Q, and in this case the real multiplication "degenerates" into two copies of multiplication by Q, in 
other words the abelian surface is no longer simple but isogenous to a product. In this particular case 
the set of F-integral subgroups (F = S'p(4, Z)) corresponds to the abelian surfaces which are actually 
isomorphic to a product. We prove more generally the following: 

Theorem 0.2 Let G, S, P^, Ab and F be as above (b < t), F' G G'(Q) arithmetic, and let Ap/^ be a 
modular subvariety of X^' for N rational symmetric, conjugate to N^. Then Ap/^ is a finite quotient of 
a product, and the set of V -equivalence classes of such modular subvarieties forms a locus in Ap' where 
the corresponding abelian varieties are isogenous to products, i.e., are not simple. If N is T -integral, 
then Ap/^ is a product, and the set of T' -equivalence classes of such modular subvarieties forms a locus 
in Ap' where the corresponding abelian varieties split while preserving the endomorphisms (but not 
necessarily the polarizations). 

An important application of the main result is to define a simplicial complex which is an analog 
for reductive groups of what the Tits building is for parabolic groups. Recall this complex T(G) is 
constructed by forming the simplicial complex of the set of all rational parabolics, partially ordered 
by the inverse of the inclusion. Taking F-equivalence classes of the parabolics gives rise to a finite 
quotient complex T{G) /F, whose vertices are in one to one correspondence with the boundary varieties 
of Ap. However, the Tits building T{G) is a rational invariant of G, not depending in any way on the 
arithmetic group F. As an analog we can define a complex 5(F) by replacing the maximal Q-parabolics 
by the F-integral symmetric subgroups incident with the maximal Q-parabolics. The arithmetic group 
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r acts on 5(r), and the quotient complex 5(r)/r is again finite, by our main result above, and the 
vertices are in one to one correspondence with the F-integral modular subvarieties which are incident 
with a rational boundary variety. In this case the complex S{T) itself is an integral, not a rational, 
object. This will be dealt with elsewhere. 

We now sketch the contents of the paper. The first paragraph is preliminary and recalls some mostly 
known facts on the classification of the rational groups of hermitian type for which we could find no 
reference. In the second we recall some notions and results from |H2|| on which the theory is based. The 
third papagraph essentially describes all arithmetic subgroups of the rational groups of hermitain type, 
in terms of maximal orders and ideals therein in division algebras D for the classical cases and in terms 
of maximal orders of exceptional Jordan algebras for the exceptional cases. Theoretically this section 
could have been dispensed with, but it does help one get the feeling for the modular subvarieties later 
on. In the fourth paragraph we introduce the notion of F-integral symmetric subgroups and derive 
the basic finiteness result. In the fifth paragraph we discuss the compactifications of the arithmetic 
quotients, and finally in the sixth paragraph we define precisely modular subvarieties and derive the 
main results above. 

Thanks I would like to acknowledge helpful discussions with Steven Weintraub which led to the 
definition of F-integral, which is more or less the central contribution of the paper. 
Notations: For an algebraic /c-group H, the group of -fC-valued points for a field extension K\k will 
be denoted Hk or H{K); similarly, for a vector space V defined over k, Vk will denote the set of 
fe-points, and for the ring of integers Ok, Vou will denote a O^-lattice in Vk- Throughout, s will denote 
the Q-rank of a Q-group G, t will denote the M-rank of G'(M), and / will denote the degree of k over 
Q, when k \s a. totally real number field fixed in a discussion. Usually d will denote the degree of a 
division algebra D, and n will denote the dimension of a D- vector space V . For a group G and a 
subset H C G, the normalizer (resp. centralizer) of H in G will be denoted Mci^) (resp. Zoi^))- 
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1 Rational groups of hermit ian type 
1.1 Notations 

We now fix some notations to be in effect for the rest of the paper. We will be dealing with algebraic 
groups defined over Q, which give rise to hermitian symmetric spaces, groups of hermitian type, as we 
will say. As we are interested in the automorphism groups of domains, we may, without restricting 
generality, assume the group is centerless, and simple over Q. We will also assume G is Zariski 
connected. Henceforth, if not indicated otherwise (occasionally G will denote a reductive group; in 
sections 2.1 and 2.2 G will be a real Lie group) G will denote such an algebraic group. To avoid 
complications, we exclude in this paper the following case: 

Exclude: All non-compact real factors of G(M) are of type S'L2(M). 

Finally, we shall only consider isotropic groups. This implies the hermitian symmetric space T> has no 
compact factors. By our assumptions, then, we have 

(i) G = Rcsi^^qG' , k a totally real number field, G' absolutely simple over k. 

(ii) D = T>i X • • • X Df, each T>i a non-compact irreducible hermitian symmetric space, / = [A; : Q]. 



1.1.1 Real parabolics 



This material is presented in detail in [BB] and [|H2|] , 1.2, so we just mention enough to fix notations. 



We work in this section in the category of real Lie groups. G will denote a connected reductive real 
Lie group of hermitian type, such that the symmetric space P = G/K is irreducible. In a well-known 
manner one fixes a maximal set of strongly orthogonal (absolute) roots, defining a subalgebra Cl C 0, 
such that A = exp(a) is a maximal M-split torus which will be fixed throughout this discussion. The 
set of strongly orthogonal roots is ordered, defining an order on A, which determines a set of simple 
M-roots Ar = {rji, . . . ,rjt}, t = rankigG = dim(^), in the M-root system $ir := $(a, g). For each 
b £ {1, . . . ,t}, the one-dimensional subtorus A^ is defined: at = C\ Ker(r7j), A^ = exp(afe). We also 

set n = Yl fl''; ^ — exp(n). The standard maximal M-parabolic, Ph, b = l,...,t, is the group 
generated by ZciAh) and N; equivalently it is the semidirect product (Levi decomposition) 

Pb = ZaiAb) X Ub, (1) 
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where Ub denotes the unipotent radical. For real parabolics of hermitian type one has a very useful 
refinement of (|l|). This is explained in detail in ]SC| ] and especially in Q, §111.3-4. First we have the 
decomposition of ZciAb) 

ZciAb) = Mb-Lb- TZb, (2) 

where Mb is compact, Lb is the hermitian Levi factor, TZb is reductive (of type Ab-i), and the product 
is almost direct (i.e., the factors have finite intersection). Secondly, the unipotent radical decomposes, 

Ub = Zb- Vb, (3) 

which is a direct product, Zb being the center of Ub- For this decomposition the groups are both 
Zariski connected and connected in the real Lie groups. The action of ZciAb) on Ub can be explicitly 
described, and is the basis for the compactification theory of ||SC| . The main results can be found in 
P], III §3-4, and can be summed up as follows. 

Theorem 1.1 In the decomposition of the standard parabolic Pb (see and ^) 

Pb = (Mb ■ Lb ■ Tib) X Zb ■ Vb, 

the following statements hold. 

(i) The action of Alb " Lb is trivial on Zb, while on Vb it is by means of a symplectic representation 
p : Mb ■ Lb — > Sp{Vb, Jb), for a symplectic form Jb on Vb- 

(a) TZb o,cts transitively on Zb and defines a homogenous self-dual (with respect to a bilinear form) 
cone Cb C Zb, while on Vb it acts by means of a representation a : TZb — * GL(Vb, lb) for some 
complex structure lb on Vb- 

Furthermore the representations p and a are compatible in a natural sense. The decomposition and 
the representations in fact are valid for the corresponding real algebraic group G and its algebraic 
subgroups. 

Finally, there is a one to one correspondence between the maximal real parabolics P (each of which 
is conjugate to a unique Pb) and the boundary components F (each of which is the image of a unique 
standard boundary component Fb), given by P < — > F, where P = Mg{F)- In particular, Pb = MciFb). 

1.1.2 Roots 

We now return to the notation used above, G = Res^qG' the Q-simple group of hermitian type, and 
introduce a few notations concerning the root systems involved. Let Sqo denote the set of embeddings 
£7 : k — > M; this set is in bijective correspondence with the set of infinite places of k. We denote these 
places by u, and if necessary we denote the corresponding embedding by a^. For each a £ Sqo, the 
group '^G' is the algebraic group defined over cr^k) by taking the set of elements g'^ , g G G' . For each 
infinite prime u we have Gk^ — {"''G')^, and the decomposition of T> above can be written 

V= HV^, := C^G')m/K(.) = {""G^/Kl^y 

We set G(j = C^G')^ and note that the discussion of the last section applies to Go- for each a. For 
convenience we now index the components Po- by i G {1, . . . , /}. For each we have M-roots 'I'i^M, 
of M-ranks ij and simple M-roots . . . , ryj^^.}, i = l,...,d. For each factor we have standard 
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parabolics Pi^. (1 < 6j < tj) and standard boundary components Fi^f,.. The standard parabolics of 
and boundary components of V are then products 

Pb(M)° = Pi,b, X--- xPrf,b„ Fb = i^i.bi X • • • X (b = (4) 

where Pi^b- C Go-^, Pb C G is a maximal Q-parabohc, and as above Pb(M)'^ = A/G',,(Pb)'^- Furthermore, 
there is a Q-subgroup Lb C G such that 

Aut(Pb)° = ib(M)°, Pb(M)° = Li,5i X . • • X Lrf,b^, (5) 

where Lj^;,. C Pi,b. is the hermitian Levi component as above. As far as the domains are concerned, any 
of the boundary components Pj^^- may be the improper boundary component Di, which is indicated 
by setting bi = 0. Consequently, Pj^o = = Gi and in (^) and @ any b = . . . , 6^), < b-i < ti 
is admissible. 

Since G' is isotropic, there is a positive-dimensional fe-split torus S' C G' , which we fix. Then 
'^S' is a maximal cr(/c)-split torus of "^G' and there is a canonical isomorphism S' — > inducing an 
isomorphism = ^{S',G') — > S' ,^G') =: ^k,a- The torus Res^^S' is defined over Q and 

contains S as maximal Q-split torus; in fact S = 5', diagonally embedded in Res^qS'. This yields 
an isomorphism ^{S,G) = ^k, and the root systems = ^{S,G), and ^k,a (for all a G Soo) are 
identified by means of the isomorphisms. 

In each group "^G' one chooses a maximal M-split torus A^^ D ^S' , contained in a maximal torus 
defined over a{k). Fixing an order on X{S') induces one also on X{'^S') and X{S). Then, for each a, 
one chooses an order on X{Au) which is compatible with that on X(^ S'), and r : X{A„) — > X[" S') = 
X{S) denotes the restriction homomorphism. The canonical numbering on Ajr^o- of simple M-roots of 
G with respect to Aa- is compatible by restriction with the canonical numbering of Aq (|BB|, 2.^ 



Recall also that each A:-root in <I>fc is the restriction of at most one simple M-root of G'(M) (which is 
a simple Lie group). Let = {/3i, . . . for 1 < i < s set c{i,a):= index of the simple M-root of 
restricting on /3j. Then i < j implies c(i,o") < c{j,a) for all o" G S. 
Each simple /c-root defines a unique standard boundary component: for 6 E {1, ... , s}, 

Ph := n ( resp. Pb := J] (6) 

which is the product of standard (with respect to Ao- and Ar^q-) parabolics Pc(b,a) C G^ (resp. boundary 
components Pc{b,a) of I'o-)- It follows that Pj C Pi for 1 < i < j < s. Furthermore, setting 



Ob ■■=Uocib,a), then 0, p. 472) 

Pb = Ob • Lb(M)0, (7) 

where Lb(M)'^ denotes the hermitian Levi component (|5|). As these are the only boundary components 
of interest to us, we will henceforth refer to any conjugates of the Pb of @ by elements of G as rational 
boundary components (these should more precisely be called rational with respect to G), and to the 
conjugates of the parabolics Pb as the rational parabolics. 

1.2 Classification 

For the convenience of the reader we sketch the classification of rational groups of hermitian type. As 
G = ResunG' for an absolutely simple G' over k we need only classify these. 
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1.2.1 Classical cases 



By means of the correspondence given by Weil in [ |VV| if G is of classical type, classifying the 
(semi) simple /c-groups of interest to us is equivalent to classifying the central (semi)simple /c-algebras 
with involution such that Aut(A, *) is of hermitian type. We now just list the cases, the possible 
fe-groups G' , the set of M-points of G' as well as of the Q-group G, and the corresponding domains. 
Let A: be a totally real number field of degree / over Q. For the bounded symmetric domains we shall 
use the notations Ip,q, Ilm Hln) IVn, V, VI. In what follows G' will be simple but not necessarily 
center less. 

O Orthogonal type 

O.l split case: G' = SO{V,h), V a A:- vector space of dimension n + 2, h a symmetric bilinear 
form such that, at all real primes i^, has signature (n, 2). 

/ 

G'(M) ^ SO{n, 2), G(M) ^ SO{n, 2)„ 2? ^ IVn x • • • x IVn, / factors. 

1=1 

0.2 non-split case: G' = SU{V,h), V a right D- vector space of dimension n, h is a skew- 
hermitian form; here Z? is a quaternion division algebra, central simple over k, and for all 
real primes, either 

* D^^m, ^ SU{W, h), h a skew-hermitian form on M", 

* D^^ M2(M), Gl ^ SO{2n - 2, 2). 

and in the first case h has Witt index [^]. Number the primes such that for z^i, . . . , i/j^ the 
first case occurs and for z^/^+i, ■ ■ ■ ■,1'f the second occurs. Then G'(]R) = SU{W^, h), and 

/i / 

G(M) ^]T5C/(H",/i)iX Yl SO{2n-2,2)i, V^ II^x ■ ■ ■x 11^ x IV2n-2 x • • • x IV2n-2 ■ 

«=/i+l factors f-fi factors 

S Symplectic type 

5.1 split case: G' = Sp{2n,k), G'{R) ^ Sp{2n,R), G(M) ^ (5p(2n,M))^, V ^ (Illn)^. 

5.2 non-split case: G' = SU{V,h), where V is an n-dimensional right vector space over a 
quaternion division algebra D, central over k, which is however now required to be totally 
indefinite, and /i is a hermitian form on V. Then G'(M) = Sp{2n,M), and 

G(M) ^ {Sp{2n,R)y, V = (Illn)-^. 

U Unitary type 

U.l split case: G' = SU{V, h), where V is an n-dimensional IT- vector space, K\k an imaginary 
quadratic extension, and /i is a hermitian form. Let for each real prime {pu,Qu) denote 
the signature of h^- Then 

G(M) - n ^^(p^^ ^ = ip^i''!''. ip'^/'i^/ • 

V 

U.2 non-split case: G' = SU{V,h), where D is a division algebra of degree d, central simple 
over K (K as in U.l) with a ir|A;-involution and V is an n-dimensional right D-vector 
space with hermitian form h. If d = 1 this reduces to U.l, so we may assume d > 2. Again 
letting {pu,qu) denote the local signatures, we have 

GiR)^llSUip,,q,), P = x...xlp,^,q,^. 
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1.2.2 Exceptional groups 

The exceptional groups can be classified by results of Ferrar as we now describe. A general reference to 



non-associative algebra used here is |Sch| . See also |FF] for an excellent survey and further references. 



Definition 1.2 For an alternative algebra with involution (2t, — ) let 7 = (71, . . . ,7„) be a diagonal 
matrix with coefficients in k, and set il(2t"',7) := {g G M„(2t) 79*7""*^ = g}- One defines the Jordan 
algebra J(2l, 7) by taking n = 3, 

J(2l,7) :=i:(2l^7)■ 

If 21 is an octonion algebra we call J(2t, 7) the exceptional simple Jordan algebra defined by 21 and 7. 
In particular, the following cases for exceptional simple Jordan algebras can occur over M: 

(i) J'^ = J(C, (1, 1, 1)) (the compact form) 

(ii) J' = vJ{<l, (1, -1, 1))V~^ , V = diag{l, i, 1), (8) 

(iii) = J(0, (1, 1, 1)) (the split form). 

There is only one M-form for the split octonion algebra O. Furthermore, for an algebraic number field 
k, there are 3* isomorphism classes of Jordan algebras, where t denotes the number of real primes of 



k. The Jordan algebras J'^, J , and J'* have the following explicit matrix realisations (see |Dr|, p. 33) 



6 X3 X2 

J" (respectively J^) = ^ g = \ X3 ^2 xi I G M, Xj G £ (respectively Xi £ O) } (9) 



X2 XX i3 
i\ iX3 X2 

= {g= \ ix3 6 ixi 



eiGM,XiGCV, (10) 



X2 ixi 

and the algebra of Definition |1.2| is given explicitly as a matrix algebra as follows: 



6 72^:^3 I3X2 

-''(21, (71,72,73)) = "( a: = I 71^3 6 73x1 I G A;, Xi G 2t )> . (11) 

71x2 72x1 ^3 

Utilizing composition algebras and Jordan algebras one can construct, with the following construction 
of Tits algebras, exceptional Lie algebras. 

Definition 1.3 Let 21 be a composition algebra over k, and ^ = J(*B,7) a Jordan algebra over 
another composition algebra ^ as in Definition |1.2|. Set: 



£(21,3) = I?er(2l) © (2to 5o) e I?er(5). 

One defines a multiplication [•, •] on L(2l, 5), which extends the [, ] products on L'er(2l) and DerQ), 
by the rules: 

(a) [•, •] is bilinear and [x,x] = for all x G £(21,^); 

(b) [•, •] restricts to the usual commutator on Z?er(2l) and Der{^), and these are orthogonal with 
respect to [•, •], i.e., [D,E] = for ah D G Der{Ql), E G DerQ)] 
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(c) For D E Der{^), E G -Der(5), D + E acts on 2to 5o by: 

[D + E,a® x] = D{a) ®x + a® E{x); 



(d) [•, •] is defined on 2to 5o by the formula: 



[a(g)x,b(g)y] = -T{x oy) < a,b> +(a *b)(g) {x*y) + -T{a -b) < x,y >, 
3 2 



where the * and < , > products are defined as in ||Di|] (in the cases which we require the definition 
simphfies somewhat and will be described below). 

This makes £;(2t,^) a Lie algebra. 

For later use we mention that (jl^) allows us to write elements in J(2t, 7) in the following way: 

3 3 

X = ^ den + ^ Xi[j, k], Xi[j, k] := jkxejk + Jjxekj, (12) 
1=1 1=1 

and the second sum is over cyclic permutations (i, j, k) of (1, 2, 3). In these terms the norm and trace 
forms are given by (see [[FF|, 4.11) 



N{x) = + dl2l3n{xi) + 7i?273"(a^2) + 7i72?3"(a;3), (13) 

r(x) = 6 + 6 + 6- (14) 

In the first formula n(a) = a • a is the norm in 21. The norm above is of course analogous to the 
determinant in a usual matrix algebra. In particular, N(x) 7^ is a neccessary and sufficient condition 
for X to be invertible in ^, i.e., A^(x) ^ <^=^ ^ye2 '^ith x • y = l,x^ ■ y = x, and the inverse of x is 
given by: 

^ = 4r-v (15) 

N{x) 

where satisfies x ■ x* = A^(x) • 1, or explicitly 

X* = ^(^jCfc - ljlkn{xi))eii + ^(7i(x-x^) - iiXi)[i, k]. (16) 

Ee There are two constructions leading to the real Lie algebra of hermitian type C6(-i4)- O'^ one 
hand there is the algebra ^^(C, J^) (see Definition pT^ ), where is isomorphic to the Jordan algebra 
5(C, (1, — 1, 1)) of Definition |1.2| and is given explicitly as a matrix algebra in (|lO|). Note that in this 
case the general definition of the algebra L(C, J^) simplifies to 

£(C,J'') ^ i-4®Der{J^), (17) 

which, identifying Jg with the right translations by traceless algebra elements Ttjh, is nothing but 

Albert's twisted = \/\lZ^^^® Der{Z), liQ) = IZ^i^^^® DerQ), as mentioned in Q, p. 62. In our 

case A = — 1 and -U = J**, and this implies the ee-form is of outer type (see [p^ ], §4 and Theorem 5 b), 
p. 70). The Lie multiplication with respect to the decomposition in ([T7| ) is given as follows. Writing 
an element of 'C'(C, J^) as x = i (8) A + D, A ^ Jq and D G Der{J^) and identifying i Jq and Jg so 
that X = ^ + D, the Lie multiplication is given by 

[A + D,A' + D'] = {D{A') - D'iA)) + {[D, D'] - [L{A), L(^')]) , (18) 
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where L{A) is left multiplication in by A (cf. [Di{| , 3.2, p. 46). It turns out that this construction 
is insufficient to describe all /c-forms for number fields k. 

The other description of e6(-i4) is as J^*), where Jj* is isomorphic to the Jordan algebra 
J(2l^, (1, —1, 1)) = J(C, (1, —1, 1)) which can be explicitly described in matrix terms as 

4 - J{3(C, (1,-1,1)) (19) 




It is then clear that = for an associative algebra D whose traceless elements form a Lie 

algebra of type 5u(2, 1). With this information we can exhibit an explicit isomorphism: 

By means of the isomorphism ( |T7D we may represent an element as a fe-linear transformation of J^, 
i.e., as an element of C (8) M3{k). Write an element in £;(£, J^) as follows: D + c ^ a + ady, where 
D G Der{€-),c € (to,a£ {J\)o,y S Mz{k). The isomorphism is given by (Q, 2.1) 

L{C,J^) (20) 
D + c^a + ady ^ D ® 1 + (c a)^ + (c ® *a), + / (y^ + *y/). 

Now we have the following result of Ferrar concerning /c-forms of eg: 

Theorem 1.4 ([ JF2| , p. 201) If L is a Lie algebra of type te over an algebraic number field k, then 

L ^ L(2tfc,J(53,7)) 



as in Definition \1.3l for some octonion algebra 21 and Jordan algebra J(*B,7) as in Definition l.i, 
with 03 an alternative k-algebra of dimension two, and 7 = ^2017(71, 72, 73) is a diagonal k-matrix. 

For our situation of fc-forms of the M-algebra eg(_x4) this means: 

Corollary 1.5 Any k-form o/eg(_i4) (with k totally real) is of the form 

where (Ek is an anisotropic octonion algebra over k and ( jf )fc is a k-form of the algebra ([7^. 
As a corollary of this we get a classification of fc-groups of hermitian Eg type: 

Corollary 1.6 Let G' be an absolutely almost simple k-group of hermitian type, in the class of struc- 
tures of type Ee. Then {G')^ ~ Aut(£;(Cfcj (<^i)fc))° with the notations of the preceeding corollary, 
where '~" means isogenous. 

Since an octonion algebra 21 over k is uniquely determined up to isomorphism by the set of real primes 
at which it ramifies, the totally definite (Cayley) algebra (Cfc is unique, and we need only apply the 
classification of A;-forms of the Lie algebra su(2, 1) to get a complete classification of fc-forms of j\, 
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and hence a classification of the /c-forms of £6(-i4)- There are essentially three cases which can occur 
(let S) denote the associative algebra with involution and S}~ the A;- form of the Lie algebra u(2, 1)): 

(i) {V,h) is a fc-vector space with hermitian form h of Witt index 1, 
represented by a matrix H, and 2)~ = {g £ End{V)\gH — Hg = 
0}. 

(ii) {V, h) is a fc-vector space with anisotropic hermitian form h, rep- ^22^^ 
resented by a matrix H, and D~ = {g £ End{V)\gH — Hg = 0}. 

(iii) D is a central simple division algebra of degree three over an 
imaginary quadratic extension K of k with a i^jfc-involution, and 
D = D. 

Considering the Tits index of these Q-groups, note that since a Q-split torus is all the more M-split, 
it follows that the set of split roots of the index of G (usually drawn white in the Tits index) are a 
subset of the split roots of G(M). This gives a simple criterion for deciding which indices may give 



rise to the given M-form. Looking now at the list of Eg indices (of outer type) in [Tl], the following 
four possibilities arise for Q-forms of ^^eC-u): ^-^^6,0' ^^H^ ^-^6,2- However, as shown in @, 

the index '^Eq\ does not give rise to a bounded symmetric domain, but rather has symmetric space 
EIV in the notation of |^. The argument is roughly as follows. If C G is the anisotropic kernel, 
of type D4, then, since dim[C/, U] = 8 for a maximal unipotent subgroup (in the maximal Q-parabolic 
Pa^ n -Pag), it follows that H C EndQ{[U, U]), a relation preserved upon tensoring with M, so that 
is still not defined over M; thus the index of G(M) is ^£"11, giving rise to the symmetric space denoted 
EIV in |H|. Hence there are only three possible Tits indices, namely ^-Eg 2 > ^-^6^1 ^-^6 fc-forms 
of e6(-i4)) s-iid it may hold that the three possibilities in (21) coincide with the three possible indices. 

E7 There are two constructions utilizing the Tits algebra leading to the real form of type tj^_25)- On 
the one hand there is the algebra £(2l, J^) = £(2l, J") (see Definition |r|), where 21 = M2(M) and 
(respectively J'^) is isomorphic to the Jordan algebra J(C, (1, — 1, 1)) (respectively is the Jordan 
algebra J(C, (1, 1, 1))) in Definition and is given explicitly as a matrix algebra in (|To| ) (respectively 
in (^)). In this case the direct sum decomposition analogous to ( p!7| ) is (]Dr[, 4.6, p. 50) 

L(2l,J^) ^ {Qlo0j^)(BDer{J^). (22) 

The multiplication is given by the rules 

(i) [a(S) A,b0B] = l[a,b]0AoB+lTr{ab)[L{A),L{B)], for a,be^o, A,BeJ''; 

(ii) [D,b0B] = b0 D{B), D G Der{j''), 6 G 2to, S G (23) 

(iii) [D,D'] = usual commutator of D,D' £ Der{J^). 
The other description of e7(_25) is as the algebra L{^^JO^{W)), where JO%{^) is the Jordan algebra 



!K3(M2(M), (1, 1, 1)), and is given as a matrix algebra by (pAl). Of course we could derive an explicit 
isomorphism as in (^) between the two. But in this case it turns out that the first description is 
sufficient to get all /c-forms. Namely, we have the following result of Ferrar: 

Theorem 1.7 ( [|F3| , Theorem 4.3) Let k be an algebraic number field k and let L be a k-form of 
the Lie algebra 67. Then 

L ^ 

as in Definition \L$ for some quaternion algebra 21 over k, and exceptional simple Jordan algebra ^ 
over k. 
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For the case at hand here, namely fc-forms of the M-algebra e7(_25)) this implies 

Corollary 1.8 Let G' be an almost absolutely simple k-group of hermitian type, of type E7, and let 
q' be the Lie algebra. Then 

where is exceptional simple such that for each real prime of k, {Zk)i^ — or J'^, and 21^ is a 
quaternion algebra over k which splits at all infinite primes v . 

There are the following possibihties over Q: 

(i) 21 is split, CJq is a Q-form of 

(ii) 21 is split, is a Q-form of J^; 

(24) 

(iii) 21 is division, ^Jq is a Q-form of J ; 

(iv) 21 is division, is a Q-form of J'^. 

There are three possible Tits indices, namely £^y3,i?y 2 ^^id E\^^ . It is rather clear that the first 
(respectively the last) case above gives rise to E^f^ (respectively to -E'y^o^), and it seems natural to 
expect the other two cases to give rise to £'7 2- 

1.3 Boundary components 

We briefly discuss the rational boundary components occuring in each of the cases. Again we tabulate 
this, giving the Tits index in each case and describing the boundary components. We also describe, 
in the classical cases, the corresponding isotropic subspaces of the vector space V . Throughout, V* 
denotes the union of P and the rational boundary components. 

0.1 The Tits index is Dn^g (for n = 2(4)), '^Dn,s (for n = 0(4)) or Bn,s (for n odd), where s is the 
Witt index of h. The corresponding diagrams are (the top diagrams are for the case s = 2, the 
lower ones giving the left ends for s = 1): 




o — • — • — • • o • • • 

where the Galois action in the left-hand diagram is present only for n = 0(4). The boundary 
components of V = G'{R)/K' are: 

- {pt} C {1-disc}*, {s = 2) 

- {pt}, {s = 1). 

0.2 The index in this case is {n even) or '^D^^l-^ (n odd), where s is the Witt index of h. 

The corresponding diagrams are (with non-trivial Galois action identifying the two right most 
vertices for n odd): 
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• — o — • o • • • — o — • — ••• — • — if 

The corresponding boundary components are II*. 2 ^ IIn.4 ^ • • • ^ IIn-2s- 

5.1 The index is Cn,n-, with the usual diagram and the following boundary components: {pt} C 
III^ C C III*.i. 

5.2 The index is Cn,i, with diagram 



-o — • o — • • • — o — • — • • • — • — • 



2s 

The boundary components are then the following: IIIn.2 3 ■ ■ • D IIIn-2s- 

U.l The index is with the diagram 

o • — o • • 



o • — o- 




s vertices 



As above, let {^1^, q^) denote the signature of hi,, then in the factor V,, of V we have the boundary 
components of the type Ip^-b,q^-b for 1 < 6 < s. Hence a flag of boundary components wih be 



Illp^-l,q^-l ^ Illp^-2,q^-2 =5 U H ^P^^ s, 



U.2 The index is in this case '^A^^J_^ ^, with diagram 



-o- 




d—1 vertices 

where there are 2s white vertices altogether. Letting the notations be as for the case U.l, we 
have the following boundary components: 

Ipi,-d,qi.-d ^ XI Ip^-2d,q^-2d ^ ' ' ' ^ Ip^-sd,q^-sd • 

We now describe briefly the parabolics in terms of the geometry of (V, h) for all the cases above. 
Fixing a maximal A;-split torus and an order on it amounts to fixing a maximal totally isotropic 
(s-dimensional) subspace Hi <^ V and a basis vi, . . . ,Vs of Hi. There are then fc- vectors Vi,...,v'g 
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spanning a complementary totally isotropic subspace H2 such that h{vi,v'j) = 5ij. Then each pair 
{vi,v'j) spans a hyperbolic plane Vi (over D), and V decomposes: 

V = Vi® ■■■ ®Vs®V', V' anisotropic for h. (25) 



Furthermore, Vi® ■ ■ ■ ®Vs = H\® H2. With these notations, for 1 < 6 < s the standard fc-parabolic 
P^' C G' is given as follows: 

PI=Mg'{<vi,...,vi,>), (26) 

where < vi, . . . ^Vh > denotes the span, a 6-dimensional totally isotropic subspace. The hermitian Levi 
factor of P^' is 

4 = AAg' {Vb+i ®---®Vs® V')/Zg' (Vfe+1 ®---®Vs®V'). (27) 

It reduces to the A;- anisotropic kernel for b = s. 

For the exceptional cases we have the following possibilities: 

16' 

• Eq: Index: '^Eq 2^ boundary components: {pt} C Bg. 

9 35 

Index: E^ i, boundary components: B5. 

• E7: Index: -Efl, boundary components {pt} C IV^ C IV^q. 
Index: -£'72' boundary components IVi C IV^q. 



2 Rational symmetric subgroups and incidence 
2.1 Holomorphic symmetric embeddings 

Recall that an injection ixi '■ T) ^ V oi symmetric spaces is said to be strongly equivariant if ij) is 
induced by an injection i : Q ^ q' o{ the Lie algebras Q (resp. q') of the real Lie group G = Aut(I?) 
(resp. G' = Aut(T>')). This is equivalent to the condition that ivij^) is totally geodesic in V with 
respect to the G'-invariant metric on V . Assuming both T) and V are hermitian symmetric, there 
exist elements ^ (resp. ^') in the center of the maximal compact subgroup K (resp. K') such that 
J = ad(^) (resp. J' = ad(^')) gives the complex structure, and the condition that ij) be holomorphic 
is 

(Hi) i o ad(0 = ad(C') ° 

For any given hermitian symmetric space P', the possible hermitian symmetric subdomains iviT^) 



have been classified by Satake and Ihara (see and |S1]). Note in particular that the above applies to 
T>N, where is a reductive subgroup of hermitian type and Pat is the associated hermitian symmetric 
space. We will refer to subgroups N C G, where G is the connected component of the automorphism 
group of V, for which P^v C P is a hermitian symmetric subdomain, as symmetric subgroups N C GQ. 
For this notion it is irrelevant whether N is reductive, semisimple or even centerless. 



2.2 Incidence over M 

In this section let G be a reductive Lie group of hermitian type such that the symmetric space D 
is irreducible, and let A C G be the maximal M-split torus (with order) defined by the maximal 
set of strongly orthogonal roots of G as in 1.1.1. Then we can speak of the standard parabolics 
Pfe, b = 1, . . . ,t, t = rank^G. We introduce the set of domains (S'D) as follows. 

^The term "symmetric" arises from the fact that in most cases, A*' can be defined in terms of closed symmetric sets 
of roots. 
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(SV) Iq,q, Iln, n even, Illn. 

With respect to a fixed Pb we consider the following conditions on a symmetric subgroup N C G 
as in section 



2.1 



1) has maximal M-rank, that is, rank^A^ = rank^G. 

2) is a maximal symmetric subgroup. 

2') is a maximal subgroup of tube type, i.e., such that Vjy is a tube domain. 
2") A^ is minimal, subject to 1). 

3) N = Ni X N2, where A'^i C Pf, is a hermitian Levi factor of Pb for some Levi decomposition. 
3') contains F boundary component. 

Definition 2.1 Let G be a simple real Lie group of hermitian type, A a fixed maximal M-split 
torus defined as above by a maximal set of strongly orthogonal roots, r]i, i = 1, . . . ,t the simple M- 
roots, Fb a standard boundary component and P^ the corresponding standard maximal M-parabolic, 
b £ {1, . . . ,t}. A symmetric subgroup N C G (respectively the subdomain V^^ C V) will be called 
incident to Pb (respectively to Fb), if A^ fulfills: 

• h<t, then A^ satisfies 1), 2), 3). 

• h = t, V ^ {£V), then A^ satisfies 1), 2) or 2'), 3'). 

• b = t, Ve {£V), then A^ satisfies 1), 2"), 3'). 

For reducible V = Vi x ■ ■ ■ x V^, have the product subgroups Nb^^i x ■ ■ ■ x Nb^^d^ where ^'atj. ^ is 
incident to the standard boundary component F^. of Vi (and Nq^i = Gi). 

This defines the notion of symmetric subgroups incident with a standard parabolic. Any maximal 
M-parabolic is conjugate to one and only one standard maximal parabolic, P = gPbg~^ for some h. 
Let A";, be any symmetric subgroup incident with Pb- Then just as one has the pair [Pb, Nb) one has 
the pair {P,N), 

P = gPbg-\ N = gNbg-\ (28) 



Definition 2.2 A pair (P, N) consisting of a maximal M-parabolic P and a symmetric subgroup A^ 
is called incident, if the groups are conjugate by a common element g as in (p^) to a pair {Pb,Nb) 



which is incident as in Definition 2.1 



The existence of the symmetric subgroups Nb was proved in the above mentioned work of Ihara and 
Satake. Let Pb, 1 < b < t (this means dim(Fb) > 0) be a standard parabolic and let Lb be the 
"standard" hermitian Levi factor, i.e., such that Lie{Lb) = W-, then 

Nb := Lb X ZciLb) (29) 

is a subgroup having the properties given in the definition, unique since Lb is unique. We shall refer 
to this unique subgroup as the standard incident subgroup. As to uniqueness, the following was shown 
in g|. Prop. 2.4. 



Proposition 2.3 If (N, Pb) are incident, there is g £Vb such that N is conjugate by g to the standard 
Nb of (fg^j, where Vb is the factor of Pb of Theorem 1.1. 
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The situation for zero-dimensional boundary components was not considered in [H2] in detail, so we 
take this up now. 

Consider first the case where V {ST^), so I? is a product of factors of types: 



Ip,q (P>9),lln (n even), IVn, V or VI. 

The corresponding subgroups A''^ are: Ip-i,q C Ip,q, Iln-i C Iln, IVn-i C IVn, ^2,4, II5 or YVg C 
V, 13,3 or lie C VI. Next note that if is incident to Pt, so Pat is incident to Ft (=pt.), then any 
other domain P^v' incident to Ft (isomorphic to the given Pat) will be the conjugate by some element 
of G fixing Ft, that is by g' G Pt. If furthermore g £ N, then g leaves Pat invariant. It follows that 
N is unique (in its isomorphism class for type V and VI) up to elements in Pt modulo those in Nt- 
Hence we must find the intersection Nt H Pt- This can be done in the Lie algebras, i.e., we must find 
n Pt. 

Ihara has shown that all the subalgebras {nt)c (with the exception of IVn.i C IVn, n even) are 
regw/arsubalgebras, i.e., are generated by the Cartan subalgebra t and the root spaces q" for a £ 'i'sym, 
where "^sym is a closed, symmetric set of roots. Similarly, {pt)c is the subalgebra generated by t and 
the root spaces for a G "^par, where ^'par is a closed, parabolic set of roots, "if par = U [A — 0], 
where C A is some subset of the set of simple roots, and for any subset H C A, [H] denotes the set 
of roots which are integral linear combinations of the elements of H. Then the intersection of {pt)c 
and {nt)c is given by 

(P*)cn(nOc = t+ ( E E 0")=t+ E 0" 

\ae'I's!/m aS'I'par / Ce<^'^ sym<~\'^ par 

From this it follows that the complement of {pt)c H {nt)c in {Pt)c is given by 

This is of course not a subalgebra, but we can determine the dimension of the parameter space of non- 
trivial conjugates of Nt incident with Pt. In other words, the homogenous space Pt/{Pt H Nt) can be 
identified with the set of symmetric subgroups incident with Pt ; its dimension is the cardinality of the 
set of roots "^par—^sym^^par- To demonstrate this consider SU (4, 1). Let ai = ei— £2, . . . , 04 = £4— £5 
denote the simple roots for 0c > we have : 

^^ym = ±(£2 - £3), ±(e2-£4), ±(£2 -£5), ±(£3-64), ±(£3 -£5), ±(£4 -£5), 

^'par = +(£j - £j), (10 of these) , ±(£2 - £3), ±(£2 - £4), ±(£3 - £4), 

so that ^paT — sym H par) = +(£i " £j)> J = 2, . . . , 5. Hencc, taking the relation ^ £i = into 
account, there are three effective parameters. Geometrically this can be seen as follows. The bounded 
symmetric domain is a four-dimensional ball, the boundary component is a point, and the symmetric 
subdomain P^v is an embedded three-ball passing through the point. Now think of the four-ball 
as embedded in via the Borel embedding; the three-ball is the intersection of IB4 C with a 
hyperplane P^ passing through the given point. There is an infinitesimal P^ of hyperplanes through 
the point, so we see three effective parameters. 

Now we turn to the embedding IVn-i C IVn, n even. If G = SO{V, h), h symmetric of signature 
(n, 2), let w E y be an anisotropic vector. Then is of codimension one, h^y± has signature (n — 1, 2) 
and Nt = Ng{v~^). On the other hand the parabolic Pt is the stabilizer of a (maximal) two-dimensional 
totally isotropic subspace I G V. Then V splits off two hyperbolic planes Hi, H2, and v is in the 
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orthogonal complement oi H := Hi © i?2- So the intersection Nt H Pt is just the stabilizer of u in P^, 
i.e., 

NtnPt = {g£ G\g{I) C /, g{v) ^<v>]. 

Finally we mention the case T) G (ST^)- Then Pat is a polydisc and it is easy to see that the 
intersection NtHPt is just the parabolic in Nt corresponding to the given point. Since Nt = {SL2Y, t = 
rankiftP, the parabolic is (-Pi)*, where Pi C SL2 is the standard one-dimensional parabolic. So the 
number of parameters in this case is the dimension of Pt minus t. 

We now list the sets ^sym, following Ihara, but we will use the notations of the root systems as in 



• I 



p,q- 



{ai, . . . , ap+q-i}, "i^sym — [a2, • • • , Op+g-l]- 



Q!2 + 04 + 05 + ae- Then the 

IVg 

ai 03 04 as 

a2 



• Iln: (n even). A = {ai, . . . ,a[!i],}'$sym = [02, ■ ■ • ,a[f]]- 

• IVn: {n = 2i + 1), A = {ai, . . . ,01}, (3 := a^_i + 2q!£. The following set of roots forms a 
diagram of type as indicated: 

ai 02 ■ ■ ■ 0.1-2 CKi 

P 

• V: A = {ai, . . . , ae}, Pi := 02 + "3 + 2a4 + "5 + "6, P2 
subalgebras are determined by the following sets of roots: 

l2,4 X SU(2) II5 

Pi ai 03 04 a2 U ag cti as 04 as 

P2 

• VI: A = {ai, . . . , ay}, Pi := ae + 2as + 3a4 + 2a3 + ai + 2a2, P2 ■= "5 + 2a4 + 203 + ai + 02- 
Then the subalgebras are determined by the following sets of roots: 

13,3 He 

(—02) Pi ay as as 07 ag as a4 a2 

P2 

We can also consider the converse question, i.e., given a symmetric subgroup, what is the set of 
parabolics to which it is indicent? The answer to this is easier: if dim(P) > 0, then for any other 
boundary component F' of Vjy, conjugate to F, the parabolic Pp' = N{F') is also incident to A^. These 
boundary components are in 1-1 correspondence with the zero-dimensional boundary components of 
the second factors D2 of = Vi x V2. If dim(P) = 0, then, assuming Vn is irreducible (i.e., 
T) (i5P)), then for any other zero-dimensional boundary component P', the corresponding stabilizer 
Ppi is incident with N . If P G {£T^), then we have the set of zero-dimensional boundary components 
of the polydisc. 



2.3 Incidence over Q 

We now return to the notations of section 1.1.2; G is a simple Q-group of hermitian type. The following 



definition gives a Q-form of Definition 2.2 



Definition 2.4 Let P C G be a maximal Q-parabolic, N C G a reductive Q-subgroup. Then we shall 
say that {P,N) are incident (over Q), if (P(M), A^(M)) are incident in the sense of Definition 2.2 . 



Note that in particular N must itself be of hermitian type, and such that the Cartan involution of 
G(M) restricts to the Cartan involution of A^(R). The main result of [EI2] is the following existence 
result. 
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Theorem 2.5 Let G he Q-simple of hermitian type subject to the restrictions above (G is isotropic 
and G(M) is not a product of (M) 's), P C G a Q-parabolic. Then there exists a reductive Q- 

(2) 

subgroup N C G such that (P, N) are incident over Q, with the exception of the indices „ for the 
zero- dimensional boundary components. 

We now describe the standard symmetric subgroups A^^ incident to for the classical cases. For 
details, see | H2| . We consider the vector space V with the itsymmetric/hermitian form h. In the 
notation of (2^, if the standard hermitian Levi factor L'j, is L'j, = Mg{W)/Zg{W), W = Vb+i © • • • © 
Vs © V in the notations used there, then for 5 < s or c{s, o"j) < ti for some i = 1, . . . , f , 

N!,=Mg{W). (30) 

If 6 = s and c{s,ai) = ti for all i = 1, . . . , /, the boundary component is a point, and L'^ is the 
anisotropic kernel, and N'g as in (|30|) is not the standard symmetric subgroup incident to Pg as we 
have defined it. Rather, these subgroups correspond to the following constructions. We consider first 
the case where T> (<?^). Pick an anisotropic vector v £ V which is defined over k, and consider the 
subspace W = v-^, the space of vectors orthogonal to v. We describe the subgroup A''^ = Mg'{W), 
which depends on the choice of v. 

O.l: y is a fc- vector space; the subgroup N'^ gives rise to a subdomain 2?jv^ of type IVn-i C IVn. 

0.2: Here V is an n-dimensional D-vector space, and we have n odd; 11^ C ^ is of codimension 
one over D, giving rise to a subdomain of type Iln-i C Iln- 

U.l: In this case we get subdomains Ip-i,q C Ip,q. 

U.2: V is n-dimensional over D, where D has degree d over K; the subspace W gives rise to a 
subdomain of type Ip-d,q C Ip,q. Iteration of this gives subdomains of types Ip-jd,q C Ip,q, and 
for j = s the boundary component will be a point <^=^> sd = q, in which case Iq,q C Ip,q is a 
maximal tube domain and fulfills 2'). 

Finally we consider D G {^T)). In these cases, if the zero-dimensional boundary component is rational, 
then V splits into a direct sum of hyperbolic planes (no anisotropic kernel). We can define a unique 
polydisc by the prescription: letting V = ©I^^Fj be the decomposition into hyperbolic planes as above, 
set: 

K:={gGG'\g{Vi)<^Vu i = l,...,s}. 
For the individual cases this gives rise to the following subdomains: 



Iq,q: T^Ni. — Id,d X • • • X Id,d- In each of the factors Id,d we can apply the results of [HI] to get 
a uniquely determined polydisc. 

Iln, n even: In this case we get a subdomain II2 x • • • x II2, which is a polydisc, as II2 is a disc. 
Illn: In case S.l, the result is well known, giving just a polydisc. In case S.2, we get as a 



subdomain III2 x • • • x III2, and this case represents the exception in Theorem 2.5; in general 
no polydisc (defined over k) can be found in each factor. 
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3 Arithmetic groups 



By definition, an arithmetic subgroup T C G{Q) is one whicli is commensurable with q^^{GL(Vz)) H 
G(Q), for some ( <^=^> for any) faithful rational representation g : G — > GL(V), where y is a 
finite-dimensional Q-vector space, and Vz is a Z-structure, i.e., a Z-lattice such that Vz ®z Q = Vq. 
In the classical cases, it is natural to take the fundamental representation as g (more precisely the 
fundamental representation g' : G' — > GL£){V) determines g : G — > Res^(QGLD{V)), and for 
the exceptional structures, one has either representations in exceptional Jordan algebras and related 
algebras, or simply the adjoint representation. 

Consider first the classical groups. For these, D is a central simple division algebra over K, where 
K is either the totally real number field k or an imaginary quadratic extension of k, and D has a 
-R'lfc-involution. The rational vector space V is an n-dimensional right D-vector space, A = Mn{D) 
is a central simple algebra over K with a i('|/c-involution extending the involution on D by ([331). We 
have a itsymmetric/hermitian form h : V x V — > D such that 

G' = {g e GLD{V)\\/j:^y(zv, h{x,y) = h{gx,gy)} (31) 

is the unitary group of the situation. We take the natural inclusion given by (^), g' : G' — > GL£){V) 
and let the representation g : G — > Res^qGLDiV) determined by g' be our rational representation. 
We now consider Z-structures on V, for which we require an order A G D, i.e., a lattice that is a 
subring of D, and consider A-lattices C V. The analog of (^), after fixing the Z-structure on V, is 

= {<? G ^1^^ C £}. (32) 

Then Fx: C G{Q) is an arithmetic subgroup, as it is the set of elements which preserve the A-structure 
on V defined by L , which itself is a Z-lattice in the rational vector space V (viewing y as a Q-vector 
space). If, for example, Tf^i C T^, is a normal subgroup of finite index, we get an induced representation 
of r^/r^/ in L/L , where L is the sublattice of L preserved by Tj^'. This is the general formulation 
of an occurance which is well-known in specific cases. For example, if Tjr^i = T{N) C 5p(2n,Z) = F^ 
is the principal congruence subgroup of level N, there is a representation of F/F(A^) = Sp{2n,'Ij/N7j) 
in (Z/iVZ)2" (= £/£'). 

Now consider the exceptional groups. In the case of Eq we have the 27-dimensional representation 
in the exceptional Jordan algebra ^, while in the case of we have the 56-dimensional representation 
in the exceptional algebra of 2 x 2 matrices over 00. In both cases we can also use the adjoint 
representation, so we require a Z-structure on the Lie algebra itself. Such can be readily constructed, 
utilizing the Tits algebras, from lattices in the constituents, composition algebras and (exceptional) 
Jordan algebras. 

After these introductory remarks we proceed to give a few details, which in particular allow us to 
give some relevant references in each case. We start by discussing orders, then describe the arithmetic 
groups these give rise to. 



3.1 Orders in associative algebras 

A general reference for this section is 0]. We first fix some notations, is a totally real Galois 
extension of degree / over Q, and Ok will denote the ring of integers in k. D will denote a division 
algebra (skew field), central simple of degree d over K, with a K\k involution (K = k for involutions 
of the first kind, and K is an imaginary quadratic extension of k for involutions of the second kind). 

^This is what W. Baily utilized in his beautiful paper |Ba|]. 
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V denotes an n-dimensional right i?- vector space, so that Hom£i{V, V) = Mn{D). A = Mn{D) is a 
central simple algebra over K with involution extending the involution on D by 

M ^ M*, where {M*)ij = mji, for M = (niij), (33) 

where "-" denotes the involution in D. {V, h) is a it-hermitian space with it-hermitian form h (with 
respect to the involution on D). Hence [D : K] = d?, [A : K] = (nd)"^ =t^,t = nd. 

Let -F be a number field, for example F = K,k as above, and let W be an F-vector space. A 
full Op-lattice £ in is an Oj7-module, finitely generated, such that F ■ C = W. Usually we work 
with full lattices and delete the word full. If W is an F-algebra, then an C'jT'-lattice C is an Op- order, 
if £ is a subring of VF. In particular in W = D, an Oj^-order is a (full) lattice which is a subring. 
Let A C D denote an order in D, and let V be an n-dimensional vector space over D. Then a (full) 
A-lattice in 1/ is a A-module with Ai ■ D = V; li again A is the algebra Mn{D), then a A-lattice 
in A is a A- order, if it is a subring of A. 

Let an Oi^-lattice C C Ahe given. C determines a right (respectively left) Oi^-order: 

Or{C) = {x£AC-xCC}, (respectively Oi{C) = {x e A x ■ C C £}). (34) 

If £ is a A-lattice, then Or{C) and Oi{C) are A-orders. If an O^^-order O C ^ is given, and C C A 
is a lattice with O = Or[C) (respectively Oi{C)), then one also calls C an O-lattice, and says that C 
and O are associated. An element a € ^ is called integral, if its characteristic polynomial has integer 
coefficients, Xa £ Cf[^]- It is a basic result that every element a G O is integral for any O^^-order O 
in A. An order O is maximal, if it is not properly contained in any other order. It is a basic fact that 
maximal orders exist in D and in A, and that any order is contained in a maximal one ([0, 10.4). 
One has the following description of maximal orders in A: 

Theorem 3.1 ([0, 21.6) Notations as obove, let A C D be a fixed maximal Op-order in D, and let 
Ai be any (full) right A-lattice in V. Then Hom^{M,M) is a maximal Op-order in A, and for any 
maximal Op-order O in A, there exists a (full) right A-lattice M GV with O = Homi^(J\f ,N). 

The following result of Chevally describes maximal orders in associative algebras. 

Theorem 3.2 ([^, 27.6) Let A <Z D be a maximal Ox-order in D; for each right ideal J C A, set 
A' = Oi{J). Then every maximal order of A = Mn{D) is of the form 

/A ••• A J-i \ 

\ J ■■■ J A' J 

for some right ideal J, and for each J, the lattice Oj above is a maximal order. 

In other words, to give a maximal order in A is the same as giving a maximal order A C D, together 
with a right ideal J C A, i.e., the same as giving a pair (A, J). In particular if the class number 
h{A) = 1 (note that h{A), which is defined as the number of left A-ideal classes, is also equal to the 
number of right A-ideal classes, see [Q, Ex. 7 iii), p. 232), then up to A-isomorphism there is a 1-1 
correspondence between isomorphism classes of maximal orders in D and A. 
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3.2 Orders in Jordan algebras 



First recall the result on orders in the (definite) Cayley algebra from |BS|. Let cq, • • • , 67 be the base 
of Cq given as follows 

= eoQ + eiQ + . . . + ejQ, with center eoQ and relations: 
Gi ■ Ci+i = Cj+s, Cj+i • ei+3 = ei, 6^+3 • = Cj+i, ef = -eo, i G Z/7Z. 



Define 



Then 



M:={x = Y, e Z, - Cj G ^ e 2Z}. (35) 



Lemma 3.3 ( [|BS| , 4.6) M is a maximal order in €q, and any other maximal order is isomorphic 
to M. 



(In [|BS[ the authors call M an octave-ring: a subring of containing 1, on which the norm form is 
integral, and maximal with these properties; we just call M a maximal order.) 

A general reference for the remainder of this section is p^a[| . Let R he a commutative ring. A 
Jordan algebra over R is an i?-module which is commutative and satisfies the relation 



{x"^ ■ y) ■ X = x"^ ■ {y ■ x), 'i^. 
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Definition 3.4 Let ^ be a Jordan algebra over a number field and let Ok denote the ring of 
integers in K. A full Ox-lattice £ C ^ is an order, if £ is a Jordan algebra over Ok- 

An element x G ^ is integral, if the characteristic polynomial is integral, i.e., if N{x), Q{x) and T{x) 
are integral (see [Q, pp. 91, also Q, Chapter VI, for details). Let C be an order in ^, and x G £; 
then Ok[x\ C >C is an associative subalgebra, hence finitely generated, so x is integral (yRa|] Prop. 1, 
p. 19). Conversely, any integral element of 5 is contained in an order [loc. cit. Prop. 2). 

Once again it is a basic fact that maximal orders exist (loc. cit. Thm. 2) and that an order is 
maximal if and only if it is maximal locally everywhere {loc. cit. Lemma 1). A maximal order £ C 3 
is said to be distinguished, if £ is a maximal lattice of integral elements. For example, if O C O is a 
maximal order in an octonion algebra, then J{0,j) C J{0,j) (notations as in |]^) is a distinguished 
maximal order. Conversely, for 7 = 1, 



Proposition 3.5 (|Ra|, Prop. 5, p. 115) If Z = J{'^K-,^) is the exceptional Jordan algebra over 
the totally indefinite octonion algebra Ok , then any distinguished order V C ^ is of the form J{0, 1) C 
5, with O a maximal order in Ok- 



This may be considered in some sense as an analogue of Theorem 3.1 for orders in exceptional Jordan 
algebras. 

3.3 Lattices in Tits algebras 



Let 21 be a composition algebra over K, and 3 = J(2t', 1) a Jordan algebra as in L2 over a second 
composition algebra 2t'. For a totally indefinite octonion algebra over K, 21', and a maximal order 
A' C 21', then, as we have seen (Proposition |3.5|) , £ = J(A', 1) is a distinguished order in 2 and 
conversely. More generally it is easy to see: 

Lemma 3.6 Let A C 21 6e a maximal order in the composition algebra 21. Then J(A, 7) is a maximal 
order in the Jordan algebra J(2t, 7) of Definition [77 
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Proof: L := J(A,7) is clearly a Jordan algebra over Ok, hence it is an order in ^. To see it is 



maximal, the method of [Ra] can be used. Let Li = {a £ 0_ft-|a[j, A;] G C} (notations as in (12)); this 
is a lattice in Ox, and, as can be checked, is the lattice A which we started with. If C is not maximal, 
then £ ^ and the corresponding L[ will be an O/^-lattice in 21' with Li ^ L'^, contradicting the 
maximality of A. □ 

Let A C 21 be a maximal order and £ C -5 a maximal order. Consider the Tits algebra L(2t, 5) 



of Definition L3, Recall that the construction of Tits algebras requires, in addition to the algebras 21 
and 5, also the Lie algebras Z?er(2t) and Der{^). If we have maximal orders A C 21, £ C -J, then we 
define: 

Der{A) := {D G Der{^^) D{A) C A}, Der{C) := {D G DerQ) D{C) C £}. 

Since we know that L'er(2t) is a Lie algebra of type G2 and Der{^) is a Lie algebra of type -F4, we 
are asking for Z-structures on these Lie algebras. Clearly Der(A) and Der{C) are lattices in the 
corresponding Lie algebras, which are furthermore closed under the Lie bracket. It then is natural to 
consider the following lattice in the Tits algebra: 

Aa,£ :=^er(A)eAo«)/:oel)er(£), (36) 

and the corresponding arithmetic group it defines (for G = Aut(Xj(2l, ^))'') 

Ts,c ■■= {g G G|ad(ff)(AA,£) C Aa,£}, (37) 
where G is acting by means of the adjoint representation on £;(2t, ^). 
3.4 Arithmetic groups — classical cases 

In this subsection G' will denote an absolutely (almost) simple fc-group {k a totally real number field) 
which we assume is classical, G = Rcs^qG' the Q-simple group it defines, which we assume is of 
hermitian type. We let g' : G' — > GLj:){V) be the natural inclusion and g : G — > ReSk\QGL£){V) 
be the natural representation of G defined by g' . Fix a maximal order A G D, and let £ C V be 
a A-lattice (which is in particular a Z-lattice of the underlying Q- vector space). As above, Or{C) 
(respectively Oi{C)) will denote the right (respectively left) order of £, given by the equation (^). 
First of all, we have the arithmetic subgroup GL^{C) C GL£){V), and we define the subgroup 

r'z: := {9 G G'ik)\gig)iC) C £} = g-'iGLM) C G'{k), 

and similarly Tc C G(Q). By definition these are arithmetic subgroups of G'{k) and G(Q), respectively. 
Let us see how this is related to the orders Or{C) and Oi{C). By Theorem |3.2| Or{C) is of the form 
Oj for some right ideal J C O. Our central simple algebra is in this case A = Mn{D), and Or{C) is a 
maximal order in A. Recall how the group G' and the algebra are related ( [|W[| , Thm. 2, p. 598). Let 
U = {z£ A\zz* = 1}, Uq the connected component of U, ((G')° =)Go := (Aut(^))°, and let C C Uq 
be the center of Uq. Then we have an exact sequence 

1 — >C — >Uo — >Go — >l. 

As a lattice in A we consider O := Or{C) and its intersection with Uq, 

Oo = on Uo. 
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Similarly, C := C D Oq is the center of Oq, and we have the sequence 



C 



On 



r' 



where V = Oq/C is the arithmetic subgroup V C G'{k), showing how the maximal orders are related 

to the arithmetic groups. In our situation here, {G')^ plays the role of Gq, while {U')^ = {{z G A z*z = 

1})^ plays the role of Uq. Let further C C {U')^ be the center. We have O = Oj for some right ideal 
J, and {Oj)o = Oj nUo plays the role of Oq. Then C n iOj)o = C is the center of (C?j)o, and we 
have sequences: 



1 - 


^ C - 


- {u'f - 


{G'f - 


1 




u 


u 


u 




1 - 


c - 


- {O.j)o - 


r' — 


1 



In this sense, maximal orders give rise to arithmetic subgroups. Viewing the Ofc-lattice £ as a Z-lattice 
gives the corresponding diagram for the Q-groups (with hopefully obvious notations) 



1 - 


c - 


^ C/0 - 


GO - 


1 




u 


u 


u 




1 - 


- z{o^) - 


^ Oo - 


- 


1 



We now describe this more precisely for the following special cases: 

a) Siegel modular groups. 

b) Picard modular groups. 

c) Hyperbolic plane modular groups. 

These are examples of Q-groups which are of both inner type (for a)) and outer type (for b) and c)), 
of split over M-type, meaning the Q-rank is equal to the M-rank (for a) and b)) and more or less the 
opposite of split over M-type (Q-rank equal to one, M-rank unbounded) (for c)). Case a) is well-known. 



b) is also to a certain extent, while c) was introduced in |H1|. 
a) Siegel case: 

with the involution * : X ^ JX^J, J 



A = M' 



2n 





1, 



- Aut(^,*) ^ P5p(2n,Q), V - 

- D = Q, a maximal order is A 

- r = PSp{2n,Z). 

The sequence above becomes: 

1 — > Z/(2) - 



Z 



2n 



1 



Z/(2) 



5p(2n,Q) 

U 

Sp{2n,'L) 



PSp{2n, ( 

U 

r 



1 
1. 



b) Picard case: 



A = Mn{K) with involution * 
quadratic. 



X ^ HX^H, H hermitian, where K\<Q is imaginary 
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— Aut(^, *) = PSU{K^, h), V = K'^, his a hermitian form represented by H. 

— D = K, a maximal order is A = Ok, Vz = O^. 

— r = PSU{Of^, h) (or PU{Of^, h), which is not simple, but is often considered anyway). 

The sequence above becomes: 

1 — y C — y SUiR-^^h) — > PSU{K'\h) — > 1 

U U U 

1 — ^ e — > SU{0^,h) — ^ PSU{0^,h) — ^ 1. 

Note that C is given essentially by Ok n U{1), which is ±1 except for the two fields K = 
Q{^/^), K = Q(\/— 3) which contain fourth (respectively third) roots of unity. 

c) Hyperbolic plane case: 

— D is a division algebra, central simple of degree d > 2 over K, with a iC|Q-involution, 
A C -D is a maximal order. 

— A = M2{D) with involution * : X *X, where *X = (xji), if X = (xij), and x denotes the 
involution in D. 

— Aut(^, *) is a Q-form of PSU{d, d), and V = D^, with a hermitian form h : V x V — > D 
which is isotropic, Vz = A^. 

— r = PSU{A'^,h). 

The above sequence becomes in this case 

1 — > C — > SU{D^,h) — > PSU{D^,h) — > 1 

u u u 

1 — ^ CnA — > SU{A'^,h) — > PSU{A'^,h) — > 1. 

As D is central simple over K, the center is as in the last case, C = Ok n C/(1), hence it is ±1 
except for the case K = Q(-v/^) and K = Q(\/— 3) as above. 



3.5 Arithmetic groups — exceptional cases 

We mentioned above that for the exceptional cases, there are (at least) two natural types of rep- 
resentations we can consider: representations in algebras derived from exceptional Jordan algebras 
(Tits algebras), and the adjoint representation. These representations correspond to the following 
fundamental weights: 



a;i(27) L03 
O O- 



0^4 

-o- 



-o- 



0^6(27) 
-O 



0^7(56) UJQ 

o o- 



-o- 



UJ4 

o- 



U3 

-o- 



a;i(133) 
-O 



o o 

0^2(78) L02 
In the case of Eq, the 27-dimensional (respectively the adjoint, 78-dimensional) representation cor- 
responds to the weights uJi and lvq (respectively to 102), while in the case of Ei, the 56-dimensional 
(respectively the adjoint, 133-dimensional) representation corresponds to the weight (respectively 
to iOi). We briefly discuss the arithmetic groups arising in this way. 
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16' 

We first consider the 27-dimensional representation. For this we assume G' has index ^i?g 2 and 
we use the model 

g' = CQ)x = VA%, © DerQ) 
(Albert's twisted £(-3)), where A < 0, A € We then choose a maximal order M C Zk and set 

Der{M) = {a G DerQ) a{M) Q M}. 

Then we may consider the lattice 

CiM)x := VxRmo © Der{M). 

This defines an arithmetic group: 

■■= {g G G'\g{g){CiM)x) C C{M)x}, 

where g is the 27-dimensional representation in C{^)x- 

Next we consider the adjoint representation. For this we utilize the lattice in the Tits algebra 
constructed in (|3^), and the corresponding arithmetic group (37). That lattice depends on the choice 
of a maximal order A in the Cayley algebra, as well as on one in the algebra 55. More explicitly, 

Theorem 3.7 Let g' be a k-form o/e6(-i4) as in Corollary l.l, G' as in Corollary \l.d[ , i.e., 

Let A C £fc be a maximal order in the Cayley algebra as above, let C C {Ji)\ be a maximal order 
in the Jordan algebra (Definition and set 



.-r(A n-!xar((^ (j)b.X = X^+x0y + Yi:X^GDer{A),\ 
Then is an Ot-lattice in the k-vector space q' . Set 

r(A,£) := {9 G G'ik) ad{g){Q[^^^^) C 0(a,£)}- 

Then T'^^ £^ C G'{k) is an arithmetic subgroup. 

Now consider type Ej. We first consider the 56-dimensional representaion. This is the situation 
considered by Baily in |Ba| ]. In this example k = <Q, and ^ is the exceptional Jordan algebra over Q, 

= = j(Cq, (1, -1, 1)) in the notation of Definition |]|, and 21q = M2(Q). Let McCqhe the 
maximal order (IsBI). This determines, as in 13. 61, a maximal order C in ^n. Also Z C Q defines the 



maximal order A = M2(Z) C M2(Q). This then gives rise to an arithmetic group (^(a,/;)) which Baily 
shows is maximal and has only one cusp. 

Again in this case we can also consider the adjoint representation. For this we again utilize the 
lattice (^), and as above, this determines an arithmetic group as in (^). This time, we need a 
lattice in the totally indefinite quaternion algebra 2t as well as one in the Jordan algebra (^i)^- More 
explicitly. 



Theorem 3.8 Let g' be a k-form o/e7(_25) as in Theorem i.e., 

g' ^ L{Qlk,Zk), {G'f = {Aut{g')f. 
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Let A C 2tfc be a maximal order in the indefinite quaternion algebra 2tfc as in section 3.1, and let 
C C Zk be a maximal order in the Jordan algebra cls in 3.4, and set: 




Then g'^^^^^ 



C g' is an O^-lattice, and 



G' 



'[a,c) ■= {9 e Aut{g') ad(5)(0(A,£)) c g^^^^)} n (G') 



is an arithmetic subgroup in G' 



{k). 



A more detailed discussion of these matters cn be found in |H3|. 



4 Integral symmetric subgroups 

Let G be a Q-simple algebraic group of hermitian type, and let A C G be a maximal M-split torus 
defined by the set of strongly orthogonal roots as in section 1.1, given the canonical order. Let S <Z A 
be a maximal Q-split torus with the canonical order, compatible with the given order on A. Let 
further Aq = {r/i, . . . ,7/^} be the set of simple Q-roots, and let Fb, Ph be the standard boundary 
components and parabolics as explained above, b = (c(6, ai), . . . , c(6, o"/)), 6 = 1, . . . , s. Finally, let 
A'^b be the standard incident symmetric subgroup (i.e., given by (p9|) if dim(Fb) > 0, and in terms 
of root systems as explained in section 2.2 for dim(i<b) = 0). Since Ab is a reductive subgroup, it is 
not true that any G-conjugate N' of Ab is already GQ-conjugate. Therefore we make the following 
definition, yielding a proper subset of the set of G-conjugates of the given Ab. 

Definition 4.1 Let G, S", Pb, Ab be given as above. A symmetric subgroup A' C G which is 
conjugate to Ab by an element of G(Q) is called a rational symmetric subgroup of G. 

The following well-known example illustrates the difference between rational and more general sym- 
metric Q-subgroups. 

Example 4.2 Let G' be the symplectic group G' = Sp(y,h), G = ReSk^qG', where 1/ is a fc- 
vector space of dimension 2n and h is skew-symmetric. If n = 2, the corresponding domain is a 
product of copies of the Siegel space of degree 2 (type IIl2)- The boundary components corresponding 
to Pi (respectively P2) are products of one-dimensional (respectively zero-dimensional) boundary 
components. Then Ai is also a product of two factors, Ai = Ai^i x Ai^2; and each A^i^j is a polydisc 
(H)-^. If we consider the universal family of abelian varieties parameterized by the domain V, say 
A — > T), we may consider the following conditions on the fibres At € A {t € D): 

1) At is isogenous to a product. 

2) At is simple with real multiplication by some real quadratic extension k'\k. 

We claim that the locus 1) is the locus of subdomains 2? a?', where A' is rational symmetric, while the 
locus 2) is the union of Vjy, where A' is a Q-subgroup conjugate to Ai, but not in G(Q). To see 
this, let us suppose A; = Q; we have the familiar description for the domains Dj^f of 2): in this case 
the standard symmetric subdomain is Si x Si C S2 (given by the diagonal 2x2 matrices), and it is 
conjugated (in Gr) by the matrix 
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for a real quadratic extension k'\Q. More precisely, the subdomains are given by the equations 



c,d,e) 





^ Til 


Tl2 \ 










V ^12 


T22 ) 



aril + 6ti2 + cr22 + d(rf2 - niT22) + e = o| 



(38) 



for some integral tuple (a, 6, c, d, e) with c,d = 0{p) for some prime p. Then the discriminant is 
A = 6^ — 4oc — 4de, and the field fc' = Q(\/A) is the field mentioned above; the element w S O^' 
can be taken here, for example, as w = , yielding a Humbert surface with a = 1, d = e = 0. 

The standard symmetric subgroup = SL{2,Q) x SL{2,Q) gets conjugated onto groups = SL{2,k') 
by the elements S G Sp{A,k') C 5^(4, M). The rational boundary components of Pjv' which are 
S'L2(fc')-rational, are zero-dimensional, and are also rational boundary components of T). Note that 
the domain V^i defined by the subgroup SL{2, k') also contains one-dimensional cusps of the domain 
P, the normalizers of which are defined over k' , but not over Q and these boundary components are 
consequently not rational (for either G' or A^'). 

It is clear that the union of the subdomains of given discriminant A, is the union of conjugates 
of the standard one by elements of G(Q) if and only if A is a square, giving 1). If A is not a square, 
then k' = Q(\/A), N' is conjugate to A^i by an element in G{k'), and these are the cases occuring in 
2). □ 



Next we note that the set of subgroups defined in Definition |4.1| is independent of the maximal Q-split 
torus used to define A''i; if S' is another it is conjugate in G{Q) to S, and A^' will be rational with 
respect to S exactly when it is so with respect to S' . For a fixed the set of rational symmetric 
subgroups conjugate to N is naturally identified with Ti = G{Q)/N{Q). Since G(Q) acts on Ti. so 
does any arithmetic subgroup F C G{Q) and one can consider the double coset space F\7^. By 
definition, A'^^ and A^^ will be in the same F-orbit if g{g')~^ G F, so the orbits are determined by the 
denominators occuring in g and in {g')~^, respectively. In the example above, for each prime p, the 
group A'a, a = lies in a seperate F-orbit. In particular there are in general infinitely many orbits. 
It turns out that the following definition gives a convenient notion. For b < t (by which we mean 
dim(Fb) > 0) let A^'b = A''b,i x A''b,2 be the decomposition above, and for A^ = A^^, let A^ = A^i x Af2 
denote the corresponding decomposition. If 6 = t we set A'^b = -^b,i7 N = Ni. 

Definition 4.3 Let G, S, Pb; be fixed as above, F C G(Q) arithmetic. A rational symmetric 
subgroup N C G, conjugate to A'^b hj g e G(Q), A^ = A'^^ := gN\yg~^ will be called T-integral 
(respectively strongly T-integral), if 



A^i nF 

for the element g above. 



9{N^,i n T)g~^ (respectively A^ n F = ^(A^b n F)^-^) 



For b = t, both notions coincide, otherwise strongly F-integral implies F-integral. For our purposes, 
the weaker notion will be most important. Note that since A^ = gN^^g^^ the conditions are equivalent 
to 

Nb,ing-^Tg = Ni,,inr (respectively A^bn(7"^F(7 = AfbnF). (39) 

This in turn means that g~^rg is integral on A'b.i (respectively integral on A'^b)) in other words, that for 
some rational representation p : G — > GL{V) we have p^p^^ i(-^b,i H g~^Tg) C GLiVx) (respectively 
p\j^^{Ny^^~\g~^Vg) C GL{Vi))- Note that this definition depends on the choosen maximal torus, as well 
as on F. If S' = xSx~^ is another maximal Q-split torus, then A^^ = xN\3X~^ is the standard symmetric 
subgroup with respect to 5". If A^ C G is F-integral with respect to A^b (i-e., there is g £ G{Q) such 



that gNi,g-^ = N and ATb,! n g'^Tg = A^b,i n F), then A^^^^ n {gx-'^)-^T{gx-^) = N^^ ^^ D xTx'^; in 
other words when A'" is F-integral with respect to N^y, then A^ is xFx~ ^-integral with respect to A'^ 
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(with similar statements for strongly F-integral). At least in the classical cases, when G is a matrix 
group, there is a very canonical choice for A'^b) namely as a subgroup consisting of block matrices, so 
this dependence is not unreasonable. 

Let us now suppose G is classical, p' : G' — > GLoiV) the fundamental representation, p : G — > 
ReSk\Q{GL £){V)) the corresponding representation of G. We have Pb = ReSk\QPb^ = ReSk\Q^b 
and (resp. N^^) are given in terms of {V,h) by ( p6| ) (resp. by (^)). Clearly if = A^^ and 
A^b = ^fG{W), then A^ = AfcigiW)). 

Lemma 4.4 A^ = MaigiW)) is T -integral Wz = WnVz = W n p{g~'^){Vz). 

Proof: By definition A^i n T = 5Ab,i9~^ n T = c/(Ab,i n T)g~^, and this holds if and only if A^b,i n 
g~^Tg = Ab,i H T, i.e., g~^Tg meets Ab,i in the arithmetic group T. But this holds precisely when 
g^^Tg maps Wz = W CiVz into itself, and this is equivalent to Wz = W Ci p{g^^){Vz), as p{g~^Tg) 
maps p{g~^){yz) into itself, and this is the statement of the lemma. □ 

Recall also 

Definition 4.5 A lattice Vz C being given, a submodule Wz is pure, if n • x G Wz-, n ^"L x G 
Wz. 



Lemma 4.6 There is a 1-1 correspondence between rational subspaces Wq C Vq and pure Z-submodules 
Wz C Vz, given by 

Wq^WqD Vz, Wz ^ Wz <E)z Q- 
Proof: Clear. □ 



Note that the statement of Lemma 4.4 is also equivalent to g{Wz) = g{W) H Vz, and the latter is 



by Lemma 4.6, a pure submodule. This then yields: 



Corollary 4.7 There is a 1-1 correspondence between the set ofV-integral symmetric subgroups and 
pure submodules of the form giWz), with g G G(Q) and Wz C Vz the submodule above (cf. Lemma 



Proof: As we just remarked, A^ is F-integral <^=^> giWi) is pure, and for any g{Wz), g G ^(Q), 
which is pure, A^^ is clearly F-integral. □ 

Next we consider, for d = dim(l^), the representation 

d d 

R = l\p:G^GL{V), V = /\V 

Since p{Nh) = Mg{W), it follows that R{N^,) = /\'^ p{Ni,) = Mg{/\'^W) = MciWb), where W^ is 
one-dimensional in V, defined over Q, and we have slightly abused notation by denoting by Mg the 
inverse image under R of the corresponding normalizer in GL(V). Our lattice Vz produces a lattice 
in V, Vz = l\'^Vz, and F is commensurable with R~^{GLiyz))- 

We now return to the general situation; G is Q-simple of hermitian type, Pb is a standard parabolic 
and A^b is an incident symmetric subgroup, which we take to be the standard one. Since Ab is 



reductive, by |BHC|, Theorem 3.8, there exists a rational representation vr : G — > GL{V), defined 
over Q, and an element v G Vq, such that v ■ ir{G) is a closed orbit and A'b = vr~-'^(A/'GL(v)(^)- For 
example, in the classical cases, the representation R above is such a vr. We now assume that V is given 
a Z-structure Vz such that F is given by 

T = 7T-\GL{Vz)). (40) 
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Let Wb = Q{v) be the one-dimensional vector subspace spanned by v; then we may choose a primitive 
integral vector w G Wb such that A'^b = {■^GL{v){w)) ■ We consider the orbit w ■ '/r(G); as is well- 
known there is a natural isomorphism w ■ it{G) = G/N^ given by w ■ '^{g) i— > gN^- We may consider 
the lattice Vz, defining integral points n • 7r(G) — > Vz n G/N^. 

Lemma 4.8 Assume F fulfills A subgroup N given by N = is T-integral <^=^=- under the 

isomorphism w ■ ^{G) — > G/N^, N is given by an integral point gN^, i.e., w • TT{g) G Vz- 

Proof: We have the following equivalences: 

iVi n r = <7(A^b,i n r)g-^ 



n 7r-\GL{Vz)) = giNb,i n 7r-\GL{Vj^)))g-^ 
"P^" ^(N,) n GL(Vz) = vr(5)(vr(iVb,i) n GL{Vz))7T{g-') 
^ ^(iVb,i) n 7T{g-')GL{Vz)7T{g) = 7r(7Vb,i) n GL{Vz) 

7r(iVb,i) = 

^ K{G){Wh)/Z^(G){Wh)r\^{g )GL(Vz)vr(c,) 

= K{G){WY>)/z^^G){Wb) n GL(Vz) 

^ ^((7)(VFbnVz) C Vz 
w ■ TT{g) £ Vz 

where the last equivalence follows from the fact that w is primitive. The Lemma follows. □ 



Corollary 4.9 The set of T-integral symmetric subgroups is the set of subgroups corresponding to the 
integral points, 

(T-integral symmetric subgroups \ ^ Q/]\[ f] y 
\conjugate to A^b / 

Proof: This follows immediately from the proceeding Lemma. □ 



Utilizing Corollary |4.9| , we can prove finiteness of the set of F'-equivalence classes of L-integral 
symmetric subgroups, for any arithmetic subgroup T' C G{Q). Recall the basic finiteness result of 
pHC[. 



Theorem 4.10 ([ BHC |, 6.9) Let G be a reductive algebraic group defined overQ, vr : G — > GL(V) 
a rational representation defined over Q, C C V a lattice in Vq invariant under Gz, and X a closed 
orbit of G. Then X f] C consists of a finite number of orbits of Gz. 

Here G C GL{n, C) and Gz = G n Mn{Z). 

Corollary 4.11 Given G, S, Py^, and T as above (T as in ^^), there are finitely many T'- 
equivalence classes of T-integral symmetric subgroups, for any arithmetic subgroup T' C G(Q). 



Proof: Since T satisfies (|4Cl|), Corollary holds and Theorem 4.10 may be applied to F, hence 
finiteness holds for any F'. □ 

Note that under the action of F on p{G) ■ v n Vz, all orbits are bijective to F/(F n A'^b)- Let the 
orbit decomposition with respect to F' be 



T'\p{G) ■vnVz = OiU---UO, 
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Choose, in each orbit Oi, a representative Xi, and let A^^.. be the corresponding integral symmetric 
subgroup. The set {Nx^} serves as a finite set of F-integral symmetric subgroups representing all 
r'-equivalence classes of such. The following is then well defined. 

Definition 4.12 Given G, N\y, F as above, V C G'(Q) arithmetic, the class number of T'-equivalence 
classes of T-integral symmetric subgroups is the cardinality 

Ai(G,iVb,r,r') :=|r'\(G/iVbnVz)|. 

If in a discussion G and a maximal Q-split torus S C G are fixed, then A^b depends only on the integer 
6 G {1, . . . , s} {s = rankQG)0, and we will denote this class number by fib(^, T')- 



5 Arithmetic quotients 

In this section we keep the above notations. G is Q-simple of hermitian type, D = G{M)/K = 
G(M)°/iC^ the hermitian symmetric space, T C G(Q) an arithmetic subgroup. The group T acts on 
P by means of holomorphic isometrics, preserving the natural Bergmann metric. 

Definition 5.1 The quotient '■= r\D, where F C G(M) is arithmetic, is called an arithmetic 
quotient. 

If F acts without fix points, then the quotient is a smooth complex manifold, not compact in 
general. If F has fix points, then X^ has certain quotient singularities, which can be described as 
follows. Let Fi C F be a normal subgroup of finite order without elements of finite order, so that Fi 
acts freely and hence X^i is smooth. We have a Galois cover, 

Xr, Xr, (41) 

with X^i smooth, and Galois group acting, yielding the singularities of X^- It is clear that the Galois 
group, F/Fi, creates the singularities, so they are controlled by certain properties of F/Fi, such as 
the orders of the elements, etc. In particular, Xr is still smooth if F/Fi is generated by refiections, 
as the quotient is then smooth by Chevally's theorem. It is well-known that Xr is compact <^=^ G 
is anisotropic. Suppose this is the case, and that in addition F has no elements of finite order. Then, 
as Kodaira showed in 1954 as one application of his embedding theorem, X-p is a smooth projective 
variety, the canonical bundle Kx^- being ample. In this case one has Hirzebruch proportionality, which 
states that the ratios of the Chern numbers of Xy are equal to the ratios of the corresponding Chern 
numbers of the compact hermitian symmetric spaces I), and the overall factor of proportionality is 
just the volume of X-p, which is the same as the volume in P of a fundamental domain of F, where 
volume is taken with respect to the Bergmann metric. 



5.1 Satake compactification and Baily-Borel embedding 

In case G is not anisotropic, Xr is not compact. It has a topological compactification Xp, the so-called 
Satake compactification. This is constructed by putting an appropriate topology, the Satake topology, 
on P* := DU {rational boundary components} (|pH], 4.8). With the Satake topology, the action of F 



on P extends to one on D* ([BB|, 4.9), and the quotient F\P* = Xp is the sought for compactification. 
It has the following property: 



^here again with the two exceptions for b = s = t and the two exceptional domains where there are three, resp. two 
isomorphism classes of A'b 
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Proposition 5.2 ( |pB|] , 4.11) is a compact, Hausdorff space, and the complement Xp\Xr is a 
finite disjoint union Xy\Xy = Vl U • • • U Vn, with each Vi an arithmetic quotient of dimension and 
Q-rank less than that of Xr- The length k of a maximal chain V^^ C C • • • C V* is the Q-rank of 
G. 



In our discussion of the Q-hermitian symmetric subgroups in section 1.2 we determined the rational 



boundary components for each G giving rise to quotients Vi. One case of particular interest are 



the hyperbolic planes, discussed in detail in |lHl|] . We know that in this case all rational boundary 
components are zero-dimensional, i.e., points. Hence the finite union of ^.2| is a disjoint union of 
points; the number of such is just the number of cusps, defined as follows. Suppose again we have the 
fixed Q-split torus S and the standard subgroups Pb and A^'b with respect to S. 

Definition 5.3 

(i) For h G {1, . . . , s}, the number of boundary varieties, conjugate to the 6*^ standard one, is the 
cardinality 

v,{T) = |r\G(Q)/Pb( 



(ii) The number of T -cusps is the cardinality (where B is a Borel subgroup) 

h{T) = \T\G{Q)/B{Q)\. 



Note that h(T) is also the number of maximal flags of boundary varieties, and it is often given by a 
class number, hence the notation. Since in the case of hyperbolic planes (s = 1), = h(T), both 



of these are given by the results of [HI] in terms of class numbers of certain fields. More generally, the 



number of components N occurring in Proposition 5.2 is a sum N = ri + . . . + Vg, where s = Q-rank 



of G, ri, = equivalence classes of boundary components conjugate to Fb. Then 



Proposition 5.4 For any Xr, the number N of Proposition 5.i can be expressed: N = ri + . . . + rg^ 



and Tft = z^f,(r) as in Definition 5j_S 



The term Baily-Borel embedding of X^ refers to the following result. 



Theorem 5.5 ( [pB|] , 10.11, 10.12) X^ can be embedded in projective space as a normal 



rate 

variety V . If G has no normal Q_- subgroups of dimension three, then the field of rational functions 
K(y) is canonically isomorphic with the field of automorphic functions for T . 

It follows in particular that Xr is a normal, quasi-projective variety, which is even smooth if F is 
torsion free. 

5.2 Toroidal embeddings 

Recall the decomposition (of algebraic groups over M) Pb = -^b-^b^b ^b for the real parabolic, with 
the exact sequence 

1 ^ MbLb7^b Pb ^ ^^b ^ 1; 

this gives rise to a similar sequence for Fb = F n Pb (I^) > 

l^Fl-^Fb^F^^l, 

with F^ being the intersection with the Levi factor and F^ the intersection with the radical of Pb- 
Recall further that lA^^ = Z\jV\j, where 2^b is the center, and M\^L\j acts trivially on and by means 
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of a symplectic representation on Vb, while T^b acts transitively on Z]^ defining a homogenous self 
dual cone Cb C and TZ^ acts on Vb by means of complex linear transformations, see Theorem 
1.1. This then gives us the following results about the factors of Tb (and similar results hold for any 
Tf = N{F)nT): 

i) Mb(M) is compact, hence F n Mb(M) is finite. In particular, if T has no torsion, T Ci Mb(M) = e. 

ii) r n i^b(I^) is an arithmetic subgroup of Lb(M), and T Pi Lb(M)'^ =: acts on the boundary 
component Fb, with the boundary variety Wh = r2^\Fb C X^. 

iii) Let Vz = T n Vb(M). Then the group F^^ x Vz acts on F^, x Vb{R) (recah that Vb(K) has the 
structure of complex vector space), and if F is torsion free, the quotient is an analytic family of 
abelian varieties over the arithmetic quotient = T^\F\3. 

iv) ([SC], p. 248) There is an exact sequence 

1 — >T' — > Fb — ^ F" — > 1, 

— F'= subgroup of elements in Fb acting trivially by conjugation on Lie{Z\j), 

— T"= group of automorphisms of Lie{Z\j) induced by Fb; these map Cb into itself. 

The fourth point is important for the compactification theory, as one lets first F" act, then F'. A 
sketch of the construction is as follows: fix a boundary component F, rational with respect to F (i.e., 
F n N(F) is a lattice). Let E^? C Z{F)c x V{F) x F he the realisation of P as a Siegel domain as in 
||SC| , and let 1 — > F' — > Tp — > F" — > 1 be the sequence above for = N{F) n F. Furthermore, 
the objects denoted above by a subscript ?b will be denoted here by ?(-F), for example Z{F) instead 
of Zb, C{F) instead of Cb, etc. 

Proposition 5.6 (| |SC[ j, p. 249) A partial compactification along F can be constructed as follows: 

1) Let Z{F)z act on Z{F)c defining the algebraic torus Tp; do this in the fibration 

E|i} = Ef/Z{F)z C Z{F)c/Z{F)z X V{F) xF^F 

More precisely, the map Z{F)c — > Z{F)c/Z{F)z is given by exp(27riAi), . . . , exp(27riAfc), where 
one chooses a Z-base Ci, • • • ,Cfe of Z{F)z, and Aj : Z{F)c — > C is the dual base. 

2) Now compactify the algebraic torus Tp by Tp C Tp^^^y, {cTq} a T" -admissible polyhedral de- 
composition of C{F) C Z(F) (r" as in iv) above). Tp^^^y is locally of finite type, but will have 
infinitely many components corresponding to integral vectors v £ Z{F)z D C{F). The cones a a 
themselves correspond to orbits of highest codimension, i.e., to points. If aa(^ Z{F)z is spanned 
by vi, . . . ,Vk, then a a corresponds to Ai n • • • n A^, where Aj is the divisor corresponding to Vj. 

3) Glue these into by forming the fibre product 

(E|i}) x^- (Tp^^^y) 

and setting (E^ij)^„^j= interior of the closure o/E|i} in (E^iy) x-^^ {Tp{aa})- Hence (E|i}.)|o-^| 
has a fibre structure over F x V{F) with fibres Tp^^^y If Ai is the divisor corresponding to 
as in 1), then Aj = {zi = 0}, where (zi, . . . , Zk) are local coordinates on Tp. 



33 



4) Tp still acts on (E|i|)|o-^}., as follows. T" now acts freely on (E|i|)|o-^}., giving a fibre space over 
F X V{F) with fibre a finite Tp compactification, i.e., modulo T" there are only finitely many 
integral vectors, hence components, in the fibre. NowV acts on FxV{F); as Zy^ acts trivially this 
amounts to an action ofF^xiVz as in Hi) above, and this action extends to r"\(E|i})|o-^}, hence 
an open neighborhood of F will give an open neighborhood of the boundary variety W{F) = T^\F 
in Xy; this is the sought for partial compactification. 

Next one glues these partial compactifications together by means of {cra,F}, a F-admissible collection 
of polyhedral cones, one such collection for each boundary component. The main result is: 

Theorem 5.7 ( [|SC| , Main Theorem 1, p. 252) With T, D as above, for every T -admissible col- 
lection of polyhedral cones {c^a^p}, there is a unique compactification Xy (= (Xr){(j„}) which is locally 
given at each F (more precisely at W{F)) by the partial compactification above, corresponding to the 
given collection of cones. Xy is a compact Hausdorf analytic variety, which is an algebraic space. 
Furthermore, for properly chosen T-admissible collections of polyhedral cones, the compactification is 

1) a projective resolution of the Satake compactification: Xy — > X^, hence a projective variety, and 

2) smooth with Ar := Xy — Xy a normal crossings divisor. 



6 Modular subvarieties 

In this paragraph, the data G, S, D will be fixed as above, so that for each b = 1, . . . , s = rank^jG 
we have the standard boundary components Fb, the standard parabolic Pb and the standard incident 
symmetric subgroup A'b- 



6.1 Baily-Borel compactification 

Let C G be a reductive subgroup of hermitian type (this implies in particular that N is defined 
over M, and we assume the inclusion C G is also), T>n C T> the subdomain (holomorphic symmetric 
embedding) determined by N. 

Definition 6.1 The subdomain P^v C P will be said to be defined over Q, if is a Q-subgroup of G. 

Suppose a subdomain P^v C P is defined over Q, and consider an arithmetic subgroup T C G(Q) 
and the arithmetic quotient Xy = r\P. Note that for a reductive subgroup of hermitian type N C G, 
the intersection F^v := T n will be an arithmetic subgroup if and only if A^ is defined over Q, and 
this is the case if and only if 2?7v is defined over Q. Hence the arithmetic quotient ^rjv •= '^nX^n is 
defined, and clearly fits into a commutative square 







i 


i 




Xy 



Definition 6.2 A modular subvariety on Xy is a sub-arithmetic quotient Xyj^ as in (|4^), where P^v 
is defined over Q. A modular subvariety Xyj^ C Xy will be called rational (resp. T -integral), if A^ is a 
rational (resp. T-integral) symmetric subgroup as in Definition 4.1 (resp. W^. 



The embedding of (42) turns out to extend to one of the Baily-Borel embeddings, legitimizing the 
terminology subvariety. This is given by the following result of Satake. 
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Theorem 6.3 Let C P^, C P^' be Baily-Borel embeddings. Then there is a linear injection 



making the diagram 



X* ^ 

i JV 

n n 

commute and making X^^ C X^ an algebraic subvariety. 

Proof: We have an injective holomorphic embedding ^ D which comes from a Q-morphism 
p : (A^)c ^ (G')c such that p{T]\f) C T. Hence we map apply [Q, Theorem 3, and the theorem 
follows from this. □ 

Definition 6.4 We say that ^r^v ^^'^ ^ boundary variety Wi are incident, if in D* there is rational 
boundary component F with parabolic P = N{F) covering Wi, such that N and the corresponding 
parabolic P are incident. 

Note the following 

Lemma 6.5 X^^ and Wi are incident, if and only ifW* C X^^ is a maximal- dimensional boundary 
component of X^^ (if dim(Wi) > 0), resp. if and only ifWi C X^^ (if dim(Wi) = 0). 

Proof: If dim(VFj) > 0, then the groups P and N are incident if F C and F is maximal with this 
property, and if F € is rational and maximal with this property, then P and A'^ are incident. If 
dim(Wj) = and is an incident symmetric subgroup, then Wi C is a (point) rational boundary 
component, and conversely. □ 

We now consider F-integral symmetric subgroups N and arbitrary arithmetic subgroups T' C G(Q), 
let = A^nr' and consider the corresponding integral modular subvarieties they define, -^r'^ C X^i. 
As described above, the inclusion extends to the Baily-Borel embeddings Xp, C X^,. We now take 

r to be Gz for some rational representation p : G — > GL{Vi), that is F = p^^{GL(yi)) for some 
Z-structure Vz on V. Recall the notations I'bi^'), b = 1, . . . , s and Pb(^, T')) b = 1, . . . , s of Definition 
4.12 and 5.3, respectively, for the number of b^^ boundary varieties and the number of b^^ integral 



modular subvarieties, respectively. We let Wb^i, b = 1, . . . , s, i = 1, . . . , I'bi^') be the corresponding 
boundary varieties on the Satake compactification, Ybj, b = l,...,s, j = 1, . . . , /ife(r, F') the cor- 
responding F-integral modular varieties, everything on the arithmetic quotient Xp'. Then the main 
result of the paper is the following. 

Theorem 6.6 Let F be as above, F' C G{Q) arithmetic, and Xy^i C X^, the Satake compactification, 
Xp, — X^i = '}2bi^b,i- Then E := Ylbj^bj is a complete (finite, non-empty) set of T' -equivalence 
classes ofT-integral modular subvarieties, such that for each Wb,i, there is at least one Ybj incident 
to Wb^i. 

Proof: There is for each Wb^i an incident F-integral modular subvariety because for any representative 
parabolic there is a F-integral symmetric subgroup which is incident. The finiteness result Corollary 



4.11 implies that for each A^b (of which there are finitely many) there are finitely many F'-equivalence 
classes of F-integral symmetric subgroups of G conjugate to A'b, so a complete set of F'-equivalence 
classes is finite. □ 

This gives us a well-defined, non-empty, finite set of subvarieties of the Baily-Borel embedding 
Xp, C P^ for any subgroup F' C F of finite index. Furthermore these have a prescribed behavior near 
the cusps. For example, if / : D — > C is a modular form whose zero divisor Dj on Xp, contains the 
union of the integral modular subvarieties, then / is a cusp form for F'. 
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6.2 Incidence 



Consider a toroidal compactification X-p which is smooth and projective; consider what incidence 
means here. Let Wi be a rational boundary variety, and let P be a parabolic P = Mq(^-j{F) for some 
rational boundary component F which covers W^. We have the decomposition P = {MLTZ) xi ZV of 



the parabolic. Recall from the construction 5^ that the inverse image 7r~^(Wj) in Xy of Wi C X^ is 
a divisor which is a torus embedding bundle over the family of abelian varieties 'V/Vz over Wi^ . On 
the other hand, if X-pj^ is an integral modular subvariety incident to Wi, and if dim(Wj) > 0, then 
the proper transform of X^ on Xp will meet 7r~^ (Wi) in a section of the family of abelian varieties 
over Wi. In a sense, the standard one will meet in the zero-section, the others meet in certain sections 
associated with level structures (e.g. sections of torsion points). For dim(Wj) = the situation is 
slightly different. We now discuss this in more detail. 



We consider first the case that dim(Wi) > 0. Then by (29), A^b (hence any G(Q)-conjugate) has 
the form A'^i x N2, where A'^i C Pb is a hermitian Levi factor. Considering the arithmetic group ^n^^ 
acting on V^-^ x 'Dj\f^, since the product is defined over Q, the quotient Tj\f^\'DN is at most a finite 
quotient of a product itself. We assume that in fact Xpj^ is a product (we will show below in Lemma 



5.21| that for F-integral this always holds); then Xp^^ = Xi x X2, where Xi is the arithmetic 
quotient ri\P7Vi and this is isomorphic to the boundary variety Wi. It follows that X2 = Ti\T>i\i2 has 
rational boundary components which are zero-dimensional, say w £ X|, such that with respect to the 
natural inclusion i : Xr C Xr we have 

i(Xi x{w}) = Wi. (43) 

Recall further that any two hermitian Levi factors are conjugate by an element g G V, and that, 
modulo r, this means a point of the abelian variety ^V/Vz" . This is of course true for any point 
t G Wi, so we get 

Lemma 6.7 Given Wi, X^^ any integral modular subvariety incident to it. Then the proper transform 
of Xy in Xp determines a section of the family of abelian varieties of ir^^iWi) over Wi. 



Proof: Since -'^rjv integral, by Lemma 6.21 below, -'^p^ is in fact a product X^^ = X^^ x -^^p^- 
The boundary component Wi is by ( |43| ) given by a zero-dimensional boundary component w of , 
which gets modified under vr, ^'^{Xy^) = Xp^. We have fibre spaces (at least locally over Wi) 

^~\Wi)^A,^Wi, 

where Ai = Wi x Vb(I^)/rb xi H)(^) is the natural family of abelian varieties parameterized by 
Wi . Note that the zero of Vb (IR) determines a zero section uq : Wi — > Ai, t ^ the image of G 
Vb(lK) in (^i)t = H)(IK)/Ab(i)> where Ab(t) denotes the lattice at the point t, and any element 
X £ Vb(Q) determines locally a section (Tx = ctq + x. Recall that (=-^b) determined by an 
element g G Vb(Q), so the proper transform of ^p^^ in the abelian variety part of the exceptional 
locus is [^pj = (<To + g){Wi) C Ai, which is a global section. □ 

If dim(VFj) = 0, two different situations occur, depending on whether 2? is a tube domain or not. 
They are (assume for the moment that T> is irreducible) 

• I? is a tube domain, then V is trivial and there is no abelian variety; 7r^^{Wi) is a torus 
embedding. 

• V is not a tube domain, V is not trivial, and TT~^{Wi) has an abelian variety factor and a torus 
embedding factor. 
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In the first case there is not much more to say than that each irreducible component Wij of 7r~^(Wj) 
meets the proper transform of Xp in a divisor on (which gets itself blown up at the point). 

In the second case, the dimension of the abelian variety factors and of the corresponding integral 
symmetric subvarieties are given as follows: 







Iln 


V 


dim(y) 


q{p - q) 


n — 1 


16 


dim (Pat) 


qip - 1) 




8, 10, 8 



At any rate, we have the following result: 

Theorem 6.8 The proper transform of Xy on X^- is X^pf, a partial compactification for some 
T N -admissible collection of polyhedral cones. 

Proof: Let -P/v be the parabolic in A^, P the corresponding parabolic in G. Consider the decomposi- 
tions (we omit the subscript b = s) 

Pn = {MnLnTZn) X ZnVn, P = (MLTZ) x ZV. 

Then Ljsr = L is trivial (as the boundary component is a point), and there is a natural inclusion 
Z]y C Z. Letting Cat, C denote the corresponding homogenous self dual cones, we have Cn C C, and 
both inclusions are defined over Q. Finally we have Fat = TnN which implies TNnCN = CArn(Cnr). 
We know by assumption that we have a L-admissible cone decomposition of C, and since Cat C C 
is defined over Q, this gives one also for Fa/-, as follows from ||0|, Theorem 1.13. If {a} is the cone 
decomposition of C, then {ctat}, ctat := cr n Cat gives a corresponding cone decomposition of Cn, and 
the theorem just mentioned applies. This argument applies to each boundary component of Ap^, and 
it is clear that a F-admissible collection restricts to a FAr-admissible collection. □ 



6.3 Intersection 

First note the following: 

Lemma 6.9 Given X-p and two modular subvarieties Xi,X2 C X^, the intersection, if of dimension 
> 1, is again a modular subvariety. 

Proof: We are given two injections defined over Q, ii : Ni ^ G, i2 '■ N2 ^ G, and commutative 
squares 





V ^ 




i 


i 


i 


Xi - 


Xr ^ 


- A2; 



it follows that Xi n X2 is covered by Pa^i H I?Af2 with a corresponding injection ii2 : Ni n N2 ^ G, 
again defined over Q. Since Xi and X2 are modular subvarieties, V^^ and V^^ are by definition 
defined over Q, hence so is V^-^ CiV^^. It is also a symmetric subspace since V^i r\ is totally 
geodesic in V. Consequently Xi n A2 is a modular subvariety. □ 

This can be applied in particular to the integral modular subvarieties. Hence for any two integral 
modular subvarieties Xi, the intersection defines a (maybe empty) modular subvariety. As there are 



finitely many possible intersections, from the finite set of Corollary 4.11 we get a finite set of modular 
subvarieties. Note that if Xi and X2 are both integral, then also the intersection is, in the following 
sense: Let Ai = A^^ , A'2 = A^^ , then F-integral means: 

A^bi,i n F = Ab„i n giTg^\ Ab„2 n F = Ab„2 n g2rg2\ 
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and iVi n TVs = (giNy^^g^^) D {g2Ny,^g^^). Hence 



(A^bi,inAfb.,2)nr = (A^b^,! nr) n (iVb2,2 nr) 

= (A^bi,i n giTg^^) n (iVb2,2 n giTg^^) 

= (iVbi,i n iVb.,2) n [giFg^^ n 52r52"'). 

Note that adjoining these to the integral modular subvarieties implies that on an arithmetic quotient 
Xp' for r' C r of finite index, there is a well-defined, finite, non-empty set of subvarieties, all of which 
are either integral modular subvarieties incident to rational cusps or intersections of such. 

Finally consider the boundary varieties Wi and W2 to which Xi and X2 are incident. Since XiPlXs 
is a modular subvariety, it is itself an arithmetic quotient (in general a product), and has a boundary 
variety = W* n In this sense, we make the 

Definition 6.10 Let Xi and X2 be integral modular subvarieties, incident with Wi and W2, respec- 
tively. Then we say X^2 := ^1 n X| is incident to := VFj" n WI- 

Next suppose that we are given the two parabolics, say Pi and P2, which are the stabilizers of the 
boundary components Fi and Fi, of which Wi and W2 are the quotients, Wi = ri\Fi, W2 = '^2\F2- 
Assume that F^r]F2 C F* , i = 1, 2, is a maximal boundary component in F* . Under this assumption, 
the intersection Pi n P2 is a parabolic, associated with n P|. Either of the inclusions F^ n P| C 
F* determines the parabolic which is the (non-maximal) parabolic stabilizing a flag of two terms. 
Similarly, X^ D contains F^ H P| as a rational boundary component, and either of the inclusions 
X^ n X2 C X* determines a symmetric subgroup, also the stabilizer of a flag with two terms. This 
is of course just (iVi n N2) x Zg{Ni n A^'s), where Ni is the group giving rise to Pat^, covering X^. So 
we have: Ni incident with Pj, i = 1,2, P12 := Pi Pi P2 a parabohc, then (A^'i Pi N2) x Zg{Ni D N2) is 
incident to Pi2. 

6.4 Moduli interpretation 

In this section we suppose the algebraic group G comes from a moduli problem of Pel structures, and 
will discuss the moduli-theoretic description of the modular subvarieties ^rjv the corresponding 
arithmetic quotients X^. We then also briefly describe the notion of incidence from this point of view. 

6.4.1 Pel structures 

Let V be an abelian variety over C, End{V) the endomorphism ring and EndQ{V) = End{V) (E>zQ 
the endomorphism algebra. A polarization, i.e., a linear equivalence class of ample divisors giving 
a projective embedding of V, gives rise to a positive involution on EndQ{V), the so-called Rosatti 
involution: 

Q:EndQ{V) EndqiV) (45) 

If A is a central simple algebra over Q, an involution * on ^ is called positive, if tr^|Q(x ■ x*) > for 
all X G j4, X ^ 0, where tr^^iQ denotes the reduced trace. Assuming [A, *) to be simple with positive 
involution, the M-algebra AiW) is isomorphic to one of the following (see [ph2(| . Lemma 1) 

(i) i\4(M) with involution X* = *X; 

(ii) Mr(C) with involution X* = ^X, where ~ is complex conjugation; 

(iii) Mr(]HI) with involution X* = *X, where ~ is quaternionic conjugation. 
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The algebras A occuring in (i) and (iii) are central simple over M, while those of (ii) are central simple 
over C. The Q-algebra A itself is a Q-form of one of these. The central simple algebras A over Q 
are known to be the Mn{D), where D is a division algebra over Q. If the algebra A has a positive 
involution, the same holds for D. The division algebras D which can occur are also known. 

Proposition 6.11 Let D be a division algebra over Q with a positive involution. Then D occurs in 
one of the following cases: 

I. A totally real algebraic number field k; 

II. D a totally indefinite quaternion algebra over k; 

III. D a totally definite quaternion algebra over k; 

IV. D is central simple over K with a K\k involution of the second kind, where K is an imaginary 
quadratic extension of k. 

In case III the canonical involution on D is the unique positive involution, while in case II the positive 
involutions correspond to x G such that is totally negative in k. If the algebra D has an involution 
of the second kind it is easy to see that it admits a positive one. It follows from the fact that End^iV) 
is a semisimple algebra over Q with a positive involution that each simple factor is a total matrix 
algebra Mn{D), with D as in the proposition. 



(47) 



Let {A, *) be a semisimple algebra over Q with positive involution, and let 

— >GL{n,C) (46) 

be a faithful representation. Shimura considers data V = (V, C, 9) and {A, *} and defines the notion 
of polarized abelian variety of type {A, <I>, *} by the conditions: 

(i) V is an abelian variety over C, C is a polarization; 

(ii) 6 : A — > EndQ{V) is an algebra isomorphism, and for 6{x) : V — > V 
(the ~ denoting the universal cover, i.e., y is a complex vector space) one has 
9{x) = $(x); 

(iii) the involution g determined by C as in (45) coincides on 0{A) with the involu- 
tion coming from {A,*), i.e. 9{x)^ = 6{x*). 

The condition (ii) is to be understood as follows. Fixing an isomorphism 

: F = C"/A, (48) 

each a E EndQ{V) is represented by a linear transformation of C" preserving A; that is each a can be 
represented by a matrix, and 9{x) = a is the matrix corresponding to x £ A via 9. Recall also that 
a complex torus C"/A is an abelian variety if and only if there exists a Riemann form: each positive 
(1,1) form uj gives rise to a skew symmetric matrix {qij): 

uj = qijdxi A dxj, 

where the Xi are canonical coordinates on C". Hence if we fix a positive divisor C gV, it determines 
an involution as in (^) and a Riemann form Ec{x,y) on C"/A, and these are related by 

Ec{ij{a)x, y) = Ec{x, V'(a^)y), (49) 
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where for a G EndQ{V), tp{a) denotes the matrix representation for a arising from the identification 
: y = C"/A in @. 

Let {V, C, 6) be an abehan variety of type (D, *) with D a division algebra, so that {D, *) is one 
of the algebras of Proposition |6.11 . In the notations used there, put 

[A; : Q] = /, [D : K] = <f , if D is of type IV (50) 

defining the numbers / and d. Let n=dim(y); then, assuming D to be a division algebra, 2n is a 
multiple of [D : Q], i.e., 2n = \D : Q]m. Note that [Z? : Q] = / for type I, [D : Q] = 4/ for types II 
and III, while [£> : Q] = 2d^/ if D is of type IV. For the existence of {V,C, 9) of type {D, *), certain 
restrictions are placed on we assume these are fulfilled. So under the isomorphism ^, each x ^ D \s 
represented by the matrix ^{x). This makes the lattice A with V = C"'/A, tensored with Q, a (left) 
D-module, i.e., 

m 

Q:=Q-A = ^HD)-Xi (51) 

1 

for a suitable set of vectors Xi. But this is the same as saying there exists a Z-lattice M C D, such 
that 

m 

A = {'^^{ai)xi {ai,...,am) & M}. (52) 
1 

If D is central over K, then M is clearly also an Oii'-lattice in D. The integrality of the Riemann 
form can be expressed in terms of tr£)^j^: 

mm m 

EciY, ^("*)^*' Yl ^(^i)^i) = trniKiYl (^itijb*j), (53) 

1 1 i,j 

and T = (tij) E Mm{D) is a skew-hermitian matrix: 

T* = -T, (54) 
where T* denotes the matrix (tjj), where * is the involution on D. For the lattice M one has 

trD\K{MTM*) C Z. (55) 

Hence to each {V,C,9) of type (D,^,*) one gets a *-skew hermitian T € Mm{D) and a lattice 
Ai C D. To this situation there is a naturally associated Q-group. On the vector space D"^ we 
consider 

G{D,T) := {g G GL{Dn\gTg* = T}, (56) 

the symmetry group of the *-skew hermitian form determined by T. It is now easy to determine the 
M-group: 

'Type I: Sp{m,M) x • • • x Sp{m,M) (m is even) 
Type II: Spl2m, M) x • • • x Sp{2m, R) 



G(Z),r)(I 



Type III: SO* {2m) x • • • x 50* (2m) ^^^^ 
^TypelV: U{pi,qi) x ■ ■ ■ x U{pg,qg), 



where the number of factors is in each case /, and Pu + Qu = fnd, and {pu,Qu) is the signature 
corresponding to the u^^ real prime. For each u, there is a matrix W^, which trasforms into the 
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standard form, i.e., 



WuT-^'W, = (^_^^ j , / = ^ for Type I, / = m for Type II; (58) 

W,T-^Wt = -i ( ) , Type III; (59) 

W,{iT-^)W; = ( ) , Type IV. (60) 

Let T> = Vf^jjrp-^ denote the domain determined by G(-D,T)(M) (actually a particular unbounded 

realisation of this domain, see [ ph2| , 2.6). Then P = WD^, and Zy G Dy gives rise to a normalised 
period (i.e., one of the form (1, Q)) for an abelian variety, by setting Xy = YyWy, where 

m 

, / = — , Type I, I = m, Type II; (61) 



Yy = 


f Zy 


:;) 






Yy = 


f -Zy 

V Im 


l-m 




Zy 


Yy = 


( Ip. 

V '-Z. 


Zy 



, Type III; (62) 
, Type IV. (63) 
The matrix Xy determine m vectors xi, . . . Xm of C" (in a rather complicated fashion, see formulas 



(17)-(20) in [^h2(| ), which determine a lattice A = A(2;,T, A4) by the formula in equations (|5l|)-([5^) 
above. 

Note that the representation ^ contains the representations Xv= projection on the z^th real factor 
with multiplicities. For Type IV, Py + qy = md, and p,y=multiplicity of Xu while (7i^=multiplicity of 
Xy For things to work out one must therefore assume, in case of Type IV, that iT~^ has the same 
signature {py,qy) as occurs in With this restriction, the following holds: 

Theorem 6.12 ( [lSh2|l , Thm. 1) For every z e V = 2?(D,r); and every lattice Ai G D, we get a 

polarized abelian variety = C"/A(z,T, A^) of type (D,^,*), and conversely, every such V is of the 
form V = C"-/A{z,T,M) for some z £ ^?(d,t)> M d D a lattice. 

The lattice Ai C D gives rise to an arithmetic subgroup 

r = r^D^TM) = {9^G{D,T)\gMcM} (64) 

as discussed in section |3[ If one defines an isomorphism (p : Vz — > Vz' of two abelian varieties of type 
{D, $, *) as an isomorphism of the underlying varieties, such that cl)~^{C') = C and (j)9{a) = 6'{a)(j), for 
all o G -D, then one has 



Theorem 6.13 ([Sh2], Thm. 2) The arithmetic quotient Xjp = r\D(£)7^) is the moduli space of 
isomorphism classes of abelian varieties Vz = / A{z,T, M) of type {{D,^,*), (T, Ai)} , where T is 
the arithmetic group of ^^). 

Moreover, one calls two such pairs (Ti, A4i), (T2, A^2) equivalent, if ^ueM,n(D)j such that UT2U* = 6T1 
for some positive 5 £ Q and AiiU = Ai2- Equivalent pairs give rise to isomorphic families of abelian 



varieties ( [ph2| ]. Prop. 4). Summing up, *-skew hermitian matrices T G Mm{D) determine certain 
Q-groups, lattices Ai C D determine certain arithmetic groups, and the corresponding arithmetic 
quotients are moduli spaces for certain families of abelian varieties. 
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Remark 6.14 The complex multiplication by Ai describes the endomorphism ring. The automor- 
phisms determined by M are the invertible elements, i.e., Aut{V) = M* , the group of units. 



One can also accomodate level structures in this settup, introduced in | Sh3 |, cf. also | Sh4 |. This is 



done by fixing s points yi, . . . ,ys in the L'-module Q, as in (M), and s points ti, . . . ,ts of the abelian 



variety V. One requires that the map of (48) maps the yi onto the ti. More precisely. 

Definition 6.15 Let Q be a D- vector space of dimension m, and M C Q a Z-lattice. Consider a 
conglomeration: 

T:={(A^,*),(Q,T,M);yi,...,y,}, 

where {D, *) is as above, {Q, T, M) is a D-vector space with lattice M and *-skew hermitian {D- 
valued) form T on Q, and yi points in Q. This is called a PEL-type. Consider a conglomeration: 

Q:={{v,c,ey,ti,...,ts}, 

where {V,C,6) is a polarized abelian variety with analytic coordinate 9 as above and ti are points of 
finite order on V. This is called a PEL-structure. Then Q is of type 7, if there exists a commutative 
diagram 

— ^ M — > Q(M) — > Q(M)/M — > 

i fi i , (65) 

— > A — ,C"^ V — > 

satisfying the conditions 

(i) gives a holomorphic isomorphism (strictly speaking, this is the ip^^ of above); 

(ii) / is an M- linear isomorphism, and /(M) = A; 

(iii) f{ax) = ^{a)f{x), and <&(a) defines 6 (a) for every a £ D as (p7[), (ii); 

(iv) C G C determines a Riemann form Ec as in (^). 

Note that the finite set of points yi and ti come both equipped with a form; on the former the form 
T, and the Riemann form Eq on the latter. These forms are preserved under the isomorphism. 

There is a natural notion of isomorphism of abelian varieties with Pel structures. Given two Pel- 
structures Q and Q', an isomorphism (f> : V — > V is an isomorphism from Q to Q', if (j)9{a) = 9'{a)(f) 
for all a € D, and (j){ti) = t'- for all i. 

Definition 6.16 A PEL-type 7 is equivalent to a PEL-type 7 , if D = D' , * = *' , s = s' , $ and 
are equivalent as representations of D, and there is a D-linear automorphism ji of Q such that 
T'{xiJ,, yn) = T{x, y), M/i = M! ,yiiJ, = y^modM' for all i. If Q is of type T, then Q is also of type 7 
if and only if 7 and 7 are equivalent. A PEL-type 7 is called admissible, if there exists at least one 
PEL-structure of that type. 

One has an anolgue of Theorems |6.12 and 6.13| in this situation also. 



Theorem 6.17 ( ph^ , Thm. 3) For every admissible PEL-type 7 there exists a bounded symmet 



ric 



domain D ( this is the same domain as in Theorem 6.11 ) such that the statement of Theorem 6.1i 



holds in this situation, and every PEL-structure Q of type 7 occurs in this family. 
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Table 1: Rational groups for PEL-structures. 





Type II 


Type III 


Type IV 


D 


A totally 

indefinite quaternion 
algebra over Q 


a totally 

definite quaternion 
algebra over Q 


simple division algebra, cen- 
tral over K, an imaginary 
quadratic extension of Q, 
with an involution of the sec- 
ond kind. One may assume 
D to be a cyclic algebra 


d 


2 


2 


d 


dim(y) 


2m 


2m 




Tits index 




i n^^^ 7 — 12 


2 Ad) 

dm~l,s 



The types listed are absolutely Q-simple, the Q-rank is s, and this is the Witt index of the 

ibhermitian form. 



Now define a corresponding arithmetic group as follows: 

T = {geG{D,T)\Mg = M, yig = ymodM, i = l,...,s} (66) 
Then the analogue of Theorem |6.13| is 

Theorem 6.18 ( |Sh4 |, Thm. 4) Two members of the family of Theorem 6.11 corresponding to 



points zi, Z2 G V are isomorphic if and only if zi = 7(^2) for some 7 S F, T as m 
In Table || we list the data for each of the cases II, III and IV of (|57|). 

6.4.2 Modular subvarieties 

We continue with the notations above, G, S, Pb and A'^b being fixed, F an arithmetic group satisfying 
(40), and F' C G(Q) another arithmetic group. We consider the arithmetic quotient Ap/, its Baily- 



Borel embedding X^,, and a smooth toroidal embedding Xp/. Let A C G be a rational symmetric 
subgroup, A = A^ for some b = 1, . . . , s and some g E G(Q). Let us first suppose for the boundary 
point in question Fb that dim(Fb) > 0. Under this assumption we know that Ab is a product 

Ab = Lb X ZciLi,), 

and Lb(M)'' = (Aut(-Fb))''- This implies immediately that the domain T)j\[^ is also a product. 

Let ti : T>i ^ T>i X D2 be the natural inclusion, and consider the inclusion r] : I'atjj ^ "D. Then 
r]{ii{T>i)) is a symmetric subdomain, which itself has an interpretation in terms of Pel structures, 
which is a sub-PEL structure of that attached to T>. Let us now explain this for the individual cases. 
For Pel structures, only the domains of type Ip,q, Ilm IHn occur, types U.l, U.2, 0.2, S.l, S.2. 

U.l : If 6 < t, then Fb is of type Ip-b,q-bi and Pat,^ is of the type Ip-b,q-b x Ib,b- The moduli 
interpretation is complex multiplication on abelian (p-l-g)-folds. In the locus Ip-b,q-b x Ib,b; tbe 
variety A^'^'^ splits into ^^+9^26 ^ ^2b^ where the complex multiplication has signature {p—b, q—b) 
and (6,6), respectively. 

If 6 = s = Ft is a point, Pat,^ is of type Ip-i,q- Here the abelian variety A^^'^ splits off an elliptic 
curve, AP'^'^ = Ap+'?~i x A^. Since A^ has no moduli, only the moduli of contributes. 
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U.2 : If 6 < t, then Fb is of type Ip.db,q-db and Vn^^ is of type Ip-db,q-db x Idb,db- The moduU 
involved here is a degree d division algebra D, central simple over K with if|A;-involution, as 
endomorphism algebra. In the locus Ip-db,q-db x Idb,db the abelian variety A^'^'^ splits A^^'^ = 
j^P+q-2db ^ ^2d6^ ^^^^ each factor retains the endomorphisms by D. 



li b = s = t, again Ft is a point, T^n^, is of type Iq,q C Ip,q- Here the abelian variety splits as 
A'^'i X AP~'^, where the endomorphism ring on Ap~'^ is definite, and contributes no moduli. In 
this case the only moduli contributing is the modulus of A^*^. 

0.2 : If 6 < s or s < [^], then Fb = IIn-2b, T^N^, = IIn-2b x Il2b, and the splitting is evident. For 
b = t, = IIq (n even) or IIi (n odd), both of which are points. Then for n even, A't is a 
polydisc by definition, II2 x • • • x II2, and again the splitting is evident, this time as a product 
of abelian surfaces with multiplication by the quaternion division algebra D. For n odd, A't is 
of type Iln-i, and the splitting of A^"^ is as ^ ^2n-s ^ ^2^ 

S.l : li b < t, then Fb is of type Illn-b, the subdomain T>i\i^^ is of type Illn-b x Illb, A^ splits 
^„ = j^n-b ^ jf 5 ^ ^ ^ ^j^g^ pQjj^^^ ^ (nil)". Here the abehan variety splits 

into a product of elliptic curves. 



S.2 : If 6 < t, Fb = Illn- 



2b 



IIIn-2b X IIl2b, where 6 < [f ]. If 6 = s = t, P^, = (Ilh)^ , {n 



even follows from s = t). Once again, in both cases the moduli-theoretic meaning is evident. 
This explains the expression of sub-PEL structures, and we have established 



Proposition 6.19 For any subdomain T)n^, 
manner described above. 



C V, the corresponding abelian varieties split in the 



Proof: We prove a typical case and leave the verification of the other cases to the reader. We will 
do case U.2. For this we consider the matrix Yy of (|6^). We may assume that the realisation of the 
domain T) is such that for the subdomain P^r, the corresponding Zy splits, i.e., 



Zy e V 



N 









Zu2 



(67) 



where Zy^i G Vi and Zy^2 S P2 for the decomposition Pjv = T)i x T)2. In this case, Pi is of type 
Ip-jd,q-jd) while V2 is of type Ijd,jd- We need the vectors xi, . . . ,rEm determined by Xj^, given in this 
case by the formula (20) in [3h2] 



Xy 



ni 



u: 



ml 
:V 

ml 



U 



u: 



m2 
m2 



''Id 
^d 



^md 
md 



Here the vectors Xi are given by = (*n^^ • • • ^UiJ'v'^^i ■ ■ ■ *f^^) and u^^ G CP" , v^^ G C". Now from 
the particular form of our Zy, we can conclude that also the vectors Xi have a special form. Indeed, 
comparing the above with (|67|), we see that for Wy = id we have 



Xy 









Ijd 



V 



Zy^X 







^^v,2 



Zu,l \ 

ZyO 



Lq.-jd 

Ijd J 
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so that the vector *xi, for example, has the form^ 
/ \* 



// 1 \ 





1 












,(1) 

1^,2 



z, 





u,2 



\\oj \oJ 

where z^J'j denotes the k^^ column of z^j, and similarly for the other From this it follows that the 
lattice A of (52) splits, A = Ai ® A2, where Ai and A2 are orthogonal to each other, each being itself 
a normalized period matrix 



/ 1 \ 





Ai 



/0\ 



4!^ 



Z,A2 



( \ 

1 






V 1 / 



The proposition follows from this for the case that Wy = id. Finally we note that if 7^ id, this 
does not influence the reasoning above, and the splitting remains (only the polarization is no longer 
principal). In particular, the case dim(Fb) = 0, which occurs for 6 = s, sd = q^, is covered by the 
above, 

with Zy^2 e Isd,sd- n 

We now consider conjugates N = N^. The following two lemmas apply to any G as considered in 
this paper so we assume for the moment only that G is Q-simple of hermitian type, F fulfills (|^) and 
F' C G(Q) is arithmetic. 

Lemma 6.20 If g £ G{Q_), N = N^, then the modular subvariety ^r' is a finite quotient of 

a product. Consequently, for all T>n, rational symmetric, the arithmetic subvariety ^■^ the 
locus of isomorphism classes of abelian varieties which are isogenous to products, i.e., are not simple. 

Proof: Since g G G(Q), we see that is Q-equivalent to T^n^^. For I'iVjj the statement follows from 
the fact that A'b = A^i x A''2 is a product over Q, so for g G G(Q) it is likewise true for = A''^. 
Consequently, the action of F on 2? is up to a finite action a product action. The second statement 
follows from this, as on the finite cover which is a product, the splitting property follows as discussed 
above. □ 

Now suppose N is in fact F-integral, i.e., A''i n F = g{N\),-i H T)g"'^ . 

Lemma 6.21 If N is T-integral, then the discrete subgroup N CiT is in fact a product, F^r = A'^PlF = 
ri X F2, Ti C Aut{Vi), i = 1,2. 

*for convenience the transpostition t is placed to the right of the vector in this expression 



45 



Proof: We note that there is a natural inclusion NnF C grh,i9 ^ X9^h,2g ^, and since A^iPlF C NnF 
is equal to ^rb,!^"^ we get the exact diagram 

1 1 

i i 

1 NDT ^ gVy^^ig-^ X gr^,,29-^ ^ Ki ^ 1 

1 Q ^ gTy,,2g-' K2 ^ I 

1 ^ 1 ^1, 

and the splitting n F = Fi x Q follows from that of 5Ft,,i5"^ x gT\^^2g~^'- Q is a subgroup of finite 
index in gT\)^2g ^^ , and giving the injection Fi x Q ^ g^h,ig^ x g^h,2g~^ is equivalent to giving the 
injection Q ^ gTh^2g'^- n 

It may well be that is in fact F-integral if and only if X^jsr is a product, but I have no argument 
for this. At any rate, this can now be applied to derive the moduli interpretation of ^r'^ foi' ^ 
F-integral. 

Applying the two lemmas above again in the situation that G corresponds to a PEL-structure 
yields the following. 

Theorem 6.22 Let G, S, Pb, Nyj and F be as above (h < t),T' ^ G(Q) arithmetic, and let Xp'^ be a 
modular subvariety of X^' for N rational symmetric, conjugate to N]^. Then X^'^ is a finite quotient of 
a product, and the set of V -equivalence classes of such modular subvarieties forms a locus in Xy' where 
the corresponding abelian varieties are isogenous to products, i.e., are not simple. If N is T -integral, 
then Xy'^ is a product, and the set of V -equivalence classes of such modular subvarieties forms a locus 
in Xy' where the corresponding abelian varieties split while preserving the endomorphisms. 



Proof: The first statement follows immediately from Lemma 6.20| . By Lemma |6.21 , if A^ is F-integral, 
the discrete subgroup F'^ is a product, hence so is the quotient Xy>^^, giving the second assertion. We 



know by the discussion above the moduli interpretation upstairs in V, given in Proposition 5.19| . Since 



Xy'^ itself is a product, it follows that the abelian varieties also split as = A'^ x A"' where 
Tq, the modulus of A"^, defines a point in one of the factors of ^r^; while T„_g, the modulus of A^"'^, 
defines a point in the second factor. That the endomorphisms are preserved was shown above in the 
proof of |6.19 . □ 



We leave it to the reader to derive the correct result for b = t. 

Finally we briefly mention the moduli interpretation of incidence. For this, recall that one has the 
Satake compactification and the (smooth projective) toroidal compactifications. The former relate 
to degenerations of the abelian varieties as follows. A quasi-abelian variety A' is an extension of an 
abelian variety by an algebraic torus 

1 — ^ (C*)'* — >A' — >B — ^0. (68) 

Thus A' is still an abelian group. Let c denote the dimension of the abelian variety B, n = h + c 
the dimension of A' . We now suppose that Xy is a moduli space of Pel structures, and assume the 
notations used above in this case. Let be a standard boundary component of the domain D, and 
Wi a boundary variety which is covered by F^. Let n = dim(74) for the abelian varieties parameterized 
by Xr, m = dim£)(y) so that n = mg, g = f,Af and 2d^/ in the respective cases. Since Fi, has rank 
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b, it corresponds to a vector subspace W C V with dimoiW) = dim£){V) — b = m — b{b = l,...,s = 
Witt index of the form, s < [^D) hence to abehan varieties B with dim(i?) = (m — b)g and g as 
above. We observe that the sequence (^) is relevant here, with h = bg, c = {m — b)g. An extension as 
in ( pSD is far from being unique, and the precise degenerations have been constructed in many cases 
by utihzing methods from the theory of mixed variations of Hodge structures, and this can be brought 
into relation to the toroidal compactifications mentioned above, where the parameter spaces of the 
degenerations are divisors on Xr- For our purposes (|68|) is sufficient. We now consider an integral 
modular subvariety ^r^v incident with a boundary variety Wi. As we have seen above, the abelian 
varieties parameterized by Xr^^ split, in this 

— >A^ — >A' — > — >0, (69) 

and the relation to ( |68|) is obvious; the boundary varieties are the loci in X^^ where the A^ of (|69|) 
totally degenerate. 

References 

[Ba] W. Baily, An exceptional arithmetic group and its Eisenstein series, Ann. of Math. 91 (1970), 
512-549. 

[BB] W. Baily & A. Borel, Compactification of arithmetic quotients of bounded symmetric domains, 
Ann. of Math. 84 (1966), 442-528. 

[BS] F. V. d. Blij, T. Springer, Arithmetic of octaves and groups of type G^, Indag. Math. 21 
(1959), 406-418. 

[BHC] A. Borel & S. Harish-Chandra, Arithmetic subgroups of algebraic groups, Ann. of Math. 75 
(1962), 485-535. 

[Bol] N. Bourbaki, "Elements de mathematique, Groupes et algebres de Lie" , Hermann: Paris 1968 
& 1975. 

[Dr] W. Drucker, "Exceptional Lie algebras and the structure of hermitian symmetric spaces", 
Mem. AMS, 208, (1978). 

[FF] J. R. Faulkner & J. C. Ferrar, Exceptional algebras and related algebraic and geometric struc- 
tures, Bull. Lond. Math. Soc.9 (1977), 1-35. 

[Fl] J. C. Ferrar, Lie algebras of type Eg, J. of Alg. 13 (1969), 57-72. 

[F2] J. C. Ferrar, Lie algebras of type Eg, //, J. of Alg. 52 (1978), 201-209. 

[F3] J. C. Ferrar, Algebras of type E7 over number fields, J. of Alg. 39 (1976), 15-25. 

[H] S. Helgason, "Differential geometry. Lie groups and symmetric spaces," Academic Press: New 
York 1978. 

[HI] B. Hunt, Hyperbolic planes, preprint, alggeom/9504001. 

[H2] B. Hunt, Symmetric subgroups of rational groups of hermitian type, preprint, alggeom / 
9505033. 



47 



B. Hunt, "The geometry of arithmetic quotients of bounded symmetric domains", HabiUta- 
tionsarbeit, Kaiserslautern, 1995. 

S. Ihara, Holomorphic embeddings of symmetric domains, J. Math. Soc. Japan 19 (1967), 
261-302. 

N. Jacobson, "Exceptional Lie algebras". Marcel Dekker, Inc.: New York 1971. 

N. Jacobson, "Representations of Jordan algebras" , AMS Coll. Lectures, XXVII, AMS Rhode 
Island, 1974. 

D. Kazdan, On arithmetic varieties II, Isreal J. Math. 44 (1983), 139-159. 

Tadao Oda, "Convex bodies and algebraic geometry," Ergebnisse der Mathematik und ihre 
Grenzgebiete, 3.Folge 15, Springer: Berlin 1988. 

I. Reiner, "Maximal orders", Accdcmic Press: New York 1969. 

M. Racine, Arithmetics of quadratic Jordan algebras, Memoirs. AMS, 136 (1973). 

I. Satake, "Arithmetic structures of symmetric domains," Publ. of Math. Soc. Japan 14, 
Iwanami Shoten Publ.: Tokyo 1980. 

I. Satake, Holomorphic embeddings of symmetric domains into a Siegel space, Amer. J. Math. 
87 (1965), 425-461. 

I. Satake, On some properties of holomorphic imbeddings of symmetric domains, Amer. J. 
Math. 91 (1969), 289-305. 

R. Schafer, "Introduction to nonassociative algebras," Academic Press: New York 1966. 

G. Shimura, On analytic families of polarized abelian varieties and automorphic functions, 
Ann. of Math. 78 (1963), 149-192. 

G. Shimura, Moduli and fibre systems of abelian varieties, Ann. of Math. 83 (1966), 294-338. 

G. Shimura, Moduli of abelian varieties and number theory, in "Algebraic groups and discon- 
tinuous subgroups", Proc. Symp. in Pure Math. 9, AMS, Providence, Rhode Island 1966. 

A. Ash, D. Mumford, M. Rappoport, Y. Tai, "Smooth compactification of locally symmetric 
spaces," Math. Sci. Press: Brookline 1975. 

J. Tits, Classification of simple algebraic groups. In: Algebraic groups and discontinuous 
subgroups, Proc. Symp. Pure Math. Vol. 9, AMS: Providence 1966. 

A. Weil, Algebras with involution and semisimple groups, J. Indian Math. Soc. 24 (1960), 
589-623. 



48 



